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I. INTR0D1CIQRY SECTION 



SUMMARY 



Relationships to R & D Program 

The Research and Development Program in Vocational-Technical Education 
has been based on the clinical approach involving local school systems as 
partners in the try-out phases of theoretical constructs for curriculum, 
administrative patterns, teacher education, instructional material develop- 
ment, and other aspects of vocational -technical education. The Evaluation 
Systems Project is one of three in the R & D Program concerned with compre- 
hensive local programs of vocational education. 

The Evaluation Systems Project has drawn upon the staff resources avail- 
able from the five vocational teacher education services represented in 
the R & D Program. Likewise, the findings from this project have been, and 
will be made available for use or reference by the staff involved in other 
R & D projects. 

The Problem 

There has been a growing concern on the part of citizens and school 
people ’"or evaluation of their school programs of occupational education. 

In Michigan an evaluation of the state's program of vocational education 
was completed in 1963. Following this a project was conducted to develop 
and try out a systematic approach to local program self evaluation. This 
project, conducted over a period of two years with three school systems, 
resulted in the identification of procedures that worked satisfactorily, 
and which suggested some of the important elements of a system for local 
program self evaluation. It remained to be determined, however, the 
extent to which other schools of varying size a:,td varying school and com- 
munity characteristics could follow this system satisfactorily. 



- 1 - 



The problem which was posed was essentially this: Can Michigan schools 

conduct an evaluation of their own programs of vocational education, with 
emphasis on outcomes and attainment of local objectives, maximizing the 
use of local personnel and resources, and utilizing consultant leadership 
and related professional assistance? 

Scope of the Study 

The study was conducted with the cooperation of 10 school systems. 

Four of these were classed as small in size, three as medium, and three 
large. Most geographic regions in Michigan were represented. Two staff 
members were appointed by each school to constitute a leadership team. 

Most of the work of the project leader involved working with these 20 local 
leaders and the chief administrators in these systems. The project began 
in January 1966, and was completed, except for reporting and publication 
of the revised manual by January 1, 1968. Most of the work of the leader- 
ship teams and their committees was done during the academic year 1966- 
1967. 

The emphasis was on local staff and citizen involvement, curriculum 
analysis, assessment of outcomes through follow-up, and special projects 
and activities of interest to each school. The 10 schools involved the 
committee and individual study activity of 317 staff members; and 45 
advisory committees for local programs of vocational education, consisting 
of a total of 303 citizens. 

Objectives of the Study 

1. To further try-out and demonstrate a system for the evaluation of 
vocational education on the local level which originated in the 
Michigan Project on Evaluation of Local Programs of Vocational 
Education. 

2. To discover and/or devise new or improved procedures for local 
program evaluation. 
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3. To establish a working environment in which learning of evaluation 
procedures for both local school personnel and potential leaders 

at state and national levels of vocational education can take place. 

4. To identify and describe the role of the consultant in program 
evaluation. 

5. To uncover situations that could be considered as potential 
research and development centers. 

Methods Used 



The following are the principal methods used in conducting the study. 

1. Utilize consultant and advisory services of the R & D staff and 
consultants in the MDE in selecting schools, conducting local 
leadership training, developing instruments, and in other appro- 
priate ways. 

2. Conduct workshops and conferences to prepare local leadership 
teams for their role, to provide for sharing of experiences and 
reporting progress, and for informal evaluation of the developing 
system for evaluation. 

3. Make school visits, as necessary, for orientation of administra- 
tors and project leadership teams, for consultation services to 
these schools, for monitoring each local project, and for obtaining 
information relevant to the development of the evaluation system. 

4. Appoint and utilize experienced vocational teachers as research 
assistants to provide consultant services to the schools, to 
assist the cheif investigator, and to conduct special studies 
related to local program evaluation. 

5. Utilize the information and experiences gained in the project to 
revise the manual on evaluation. 

6. Analyze the records and other data from cooperating schools to 
further develop and/or revise a generalized system, and to deter- 
mine the attainment of the objectives of the study. 
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Results Obtained 



An orientation meeting was held for school administrators and prospec- 
tive leadership teams from 12 interested school systems. A three-day 
workshop was conducted for members of leadership teams from the ten 
cooperating school systems. Three project report meetings were held, with 
all 10 schools participating. A final conference was held with emphasis 
on recommendations and implementation. Several kinds of materials rela- 
ting to local program evaluation were provided the schools. All schools 
received local administrative approval before starting. 

The project leader held 54, conferences with individual school leader- 
ship teams, nearly all of these being held in the local schools. The 
research assistants conducted 24 consultative visits to local leadership 
teams. 

All leadership teams prepared local project plans, most of which con- 
tained objectives of the project, and all listed the activities to be 
conducted. Four of the schools prepared a schedule of activities. All 
schools formed an evaluation committeee of staff, either as a steering 
committee or a working committee. 

The project leader maintained a log of activities, as did also the 
local project leaders. Records were maintained in each school. Reports 
of local projects were prepared and submitted by nine of the schools. 

Highlights of Findings. Significance, and Implications 

The objectives of the project were satisfactorily attained as follows: 

1. The evaluation system was tried out in all ten cooperating schools, 
and was demonstrated successfully in nearly all of these schools. 

As a result, the following elements were established as relevant 
in a local program self evaluation (a) administrative endorsement 
and support; (b) a local leadership team; (c) a program of 
local leadership pre-service and in-service training; (d) an 
evaluation project plan; (e) a staff committee for the evaluation; 
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(f) staff time for evaluation; (g) objectives and curricular 
analysis; (h) advisory committees for assistance in the evalua- 
tion; (i) follow-up study of former students; use of consultant 
help and instructional materials in development and use of evalu- 
ation instruments; and (j) adequate reporting of activities. 

2. New and/or improved procedures for local program evaluation were 
developed. These relate to follow-up studies, advisory commit- 
tees, curricular study, and procedures of involvement. These are 
incorporated in the revised manual on evaluation. 

3. The project provided the working environment for leadership devel- 
opment of 20 local school persons and for four graduate research 
assistants. The assistants’ activities included conducting con- 
ferences for local leaders, planning and/or conducting research 
studies, and consultation regarding local studies. 

4. The role of the consultant in program evaluation was studied and 
partially described. Further wqrk is needed on this. 

5* Situations were identified that could be considered as potential 
research and development centers. 

There was substantial evidence that local school personnel assigned 
to the project made considerable professional growth. 

Strengths in the system were identified as: (a) emphasis on objectives 

and evaluation in terms of outcomes; (b) involvement of staff and citizens 
contributing to implementation of recommendations growing out of the 
studies; (c) identification of need for and use of consultant services; 

(d) development of understandings about and support for vocational educa- 
tion in the local community; and (e) the many techniques in and related 
to local program evaluation resulting from the clinical approach. The 
project leader was not successful in getting local leadership teams to 
go very far in the identification of student behavioral goals in the 
various vocational education programs in cooperating schools, and in 
taking the logical steps to measure the attainment of the behavioral goals. 
Another shortcoming was the relatively small local emphasis given to eval- 
uation of adult and post-high school vocational education. 
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The success of this project leads to two specific recommendations in 
regard to the system. The first is that steps be taken to provide state 
level leadership and support to other school systems in the state of 
Michigan so that the system may be used and improved by many school sys- 
tems desirous of improving their programs of occupational education 
through self-evaluation studies. 

The second recommendation is that the system be tried out and demon- 
strated in other states. This would help to establish its generalizability 
to similar states and school districts, as well as to states having dif- 
ferent organizational patterns such as county unit systems of school admin- 
istration. It would be desirable to determine the applicability of the 
system to evaluation of area vocational schools and centers and to commu- 
nity college programs of vocational and technical education. It may also 
be possible and would be desirable to try out the system with state proj- 
ect leadership based in a state department of education, a research coor- 
dinating unit, or other entity. 



INTRODUCTION 



Relationship to R & D Program 

The Research and Development Program in Vocational-Technical Education 
has been based on the clinical approach to teacher education and program 
experimentation. Local school systems have been involved as partners in 
the testing and try-out phases of theoretical constructs for curriculum, 
administrative patterns, teacher education, instructional materials devel- 
opment and other aspects of vocational-technical educacion. 

Administration of R and D operations is the responsibility of faculty 
and staff in the Department of Secondary Education and Curriculum under 
U.S.O.E. Contract OE 5-85-111, and with support from Michigan State Uni- 
versity and the Michigan Department of Education. 

The Evaluation Systems Project has been one of three projects in the 
R & D Program that cut across the several occupational fields in local 
vocational education programs. The two other projects have been (1) The 
Shared-Time (Dual Enrollment) Concept for Implementing Area Vocational 
Education Programs and (2) The Rural School Project. 

The Evaluation Systems Project has drawn upon the staff resources 
available from the five vocational teacher education services represented 
in the R & D Program, particularly those involved in the Hospitality 
Education, Distributive Education, and National Office Block Projects. 
Likewise, the findings from the Evaluation Systems Project have been, and 
will be made available for use or reference by the staff involved in 
other R 6c D Projects. 

Overview ' 



This report has presented first a brief summary of the Evaluation 
Systems Project. The present section gives the background of the study, 
and its objectives, design, scope, and limits. In the following section 



the methods of conducting the study are given. The central section pre- 
sents the findings and their analysis, organized mostly on the basis of 
activities conducted in local schools. The presentation is necessarily 
of a narrative and descriptive nature rather than statistical. The con- 
clusions, recommendations, and implications which follow this are organ- 
ized and stated in such a way as to be of practical value to those who 
may seek to organize and conduct local program evaluations elsewhere. 

Background 

The background for this project is strongly dominated by a national 
concern with evaluation of vocational and technical education. The 
questions of availability; of service to all age groups and levels of 
education and to all people of varying ability, background and interests; 
and of breadth of offering to meet demand for trained workers in a rapidly 
changing occupational complex and unemployment or underemployment ; all 
have called for study. These have also become the concern of people in 
local communities and in the state, in view of increasing dollar invest- 
ments in education. 

In Michigan these concerns provided the stimulus for a state-wide 

evaluation of vocational education.^ Examples of subsequent efforts in 

other states to either conduct evaluations or to study the process are 

2 3 4 k 

those of Kaufman , Lee , Eninger , Smith**, the centers at North Carolina 

^ Vocational Education in Michigan (East Lansing: Michigan Lfcate Univer- 
sity, 1963). 252pp. 

2 

Jacob J. Kaufman, "The Role of Cost-Benefit Analysis in the Eval ation 
of Vocational and Technical Education,*' (University Park: The Pennsyl- 

vania State University, 1967). 

3 

Allen Lee, Format and Criteria for Self-Analysis of State Agencies for 
Vocational-Technical Education (Berkeley: University of California, 1967). 

Max U. Eninger, The Process and Product of T & I High School Level Voca - 
tional Education in the United States — The Product (Pittsburgh, Pennsyl- 
vania: American Institutes for Research, 1965) 433pp. 

** Harold T. Smith, Education and Training for the World of Work (Kalamazoo: 
The W. E. Upjohn Institute for Employment Research, 1963). 
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State University and at The Ohio State University, and the National 
Assessment Project. Following the Michigan study, the first project 
to envelop and try out a systematic approach to local program self 
evaluation was conducted. 6 Three Michigan schools, Gaylord, Marshall, 
and Warren-Fitzgerald participated in the original evaluation project. 

The findings of this study were the basis for a recommendation that the 
project be extended to other schools in Michigan, and that the general- 
izability of the system in other states be determined. 

The project herewith reported has utilized the findings of the pre- 
vious project, and has been based to some extent on its design. It has 
sought to maximize the use of local personnel, local resources, consul- 
tant leadership, and related professional assistance. The ultimate aim 
or purpose, of course, is improvement of vocational and technical educa- 
tion. The premise is that sound program evaluation is necessary for 
bringing about improvements in program offerings and operations. To the 
extent that changes in programs are contingent on decisions by local 
citizens and educators, relevant evaluations of the existing programs 

are essential. 

Several assumptions were made prior to the design and beginning of 
the Evaluation Systems Project. These are as follows: 

—The improvement of a local program of vocational education should be 

based on an evaluation of the program. 

—School officials and citizens want to evaluate their programs, want 
help in this endeavor, and desire education in evaluation principles 

and procedures. 

-Evaluations of local programs should be the concern of those affected 
by them and of those involved in the programs. 

-Educators and citizens are primarily concerned with outcomes of pro- 
grams, when making evaluations, but with concern for the ways and 
means largely in relation to the evaluation of these outcomes. 



> Evalua ting > Vocational Education in the Public Schools, Educational 
Research Series, Number 32, (East Lansing: MichlganStateUnlverslty 

Bureau of Educational Research Services, October 1965). 57pp. 
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--There is a need for education of prospective state leaders in voca- 
tional education in the principles and procedures of local school 

program self evaluation. 

Objectives 

The objectives of the project were drawn within the framework of these 
assumptions, and patterned somewhat after the antecedent evaluation proj- 
ect already referred to. Specifically they are as follows: 

1, To further try-out and demonstrate a system for the evaluation 
of vocational education on the local level which originated in 
the Michigan Project on Evaluation of Local Programs of Vocational 
Education, 

2, To discover and/or devise new or improved procedures for local 
program evaluation, 

3, To establish a working environment in which learning of evaluation 
procedures for both local school personnel and potential leaders 
at state and national levels of vocational education can take 
place, 

4, To identify and describe the role of the consultant in program 
evaluation, 

5, To uncover situations that could be considered as potential 
research and development centers. 

This project was designed in such a way as to maximize the values 
of local school staff and citizen involvement, to place emphasis on 
goals and outcomes, and to encourage the development and implementation 
of a system for evaluation on a permanent or continuing basis. The 
broad steps are indicated, but there is provision for flexibility to 
supplement and modify these steps and the general procedures so as to 
adapt them to different sizes and types of local schools. 
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Design 



The following steps constitute the general design of the project. 

1. Utilize the R & D staff and Michigan Department of Education con- 
sultants in an advisory capacity on the project. 

2. Select 10 schools to participate in the project. Have a leader 
of the local evaluation project appointed in each of these 
schools. Provide partial support for one research associate for 
released time in each school. These two persons were to work as 
a team. 

3. Hold meetings in individual schools and a meeting of representa- 
tives from all prospective participating schools to: 

a. Discuss objectives of the project 

h. Plan for subsequent steps in getting the project underway 
c. Begin identification of areas of concern in local school 
situations and of possible activities for local evaluation 
projects. 

4. Conduct a workshop for all local teams from schools cooperating 
in the project. Utilize the Manual on Evaluation of Local 
Programs. Employ short-time consultant help for the workshop. 

5. Hold conferences in each school as follow-up of e workshop to 

help: in setting local project objectives; in developing broad 

outlines and/or steps in each local project; and in planning for 
the organization of local school staff and resources for evaluation. 

6. Encourage each cooperating school to conduct workshops and/or 
staff conferences for preparing local staff for evaluation of 
vocational education programs and for planning activities in the 
project. 

7. Appoint experienced vocational teachers as research assistants to 
serve in a consultant capacity to these cooperating schools and 



Harold M. Byram, Evaluation of Local Vocational Education Programs 
(East Lansing: Michigan State University, 1965). 81pp. 
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to assist the state project leader in related activities. Pro- 
vide intern experiences for these assistants through the project. 

8. Utilize services of specialized consultants from the Michigan 
Department of Education as needed, and to correlate with evalua- 
tion of the state program of vocational education. 

9. Hold three progress report meetings of local evaluation project 

teams • 

10. Assemble reports and other data from cooperating schools to deter- 
mine the extent to which practices have been adopted, and with 
what success, as well as to identify new or improved practices. 

11. Analyze records and other data from cooperating schools, and 
conduct depth interviews to further develop and/or evaluate a 
generalized procedure or system# 

12. Construct the role of consultant on evaluation through a percep- 
tion study, and through analysis of records kept on the evalua- 
tion project and reports from the schools. 

13. Hold final meeting of project teams and local administrators, with 

emphasis on implementation. 

14. Utilize information and experiences gained in the project to 
rsviss fcliG manual on evaluation# 

15. Prepare periodic reports and a final report on the project. 

Scope and Duration of the Project 

The memorandum of agreement with each cooperating school covered the 
fiscal year 1966-67. However, the final report meeting was held in Octo- 
ber, 1967 and the schools were allowed until the end of 1967 to submit 
their reports on the project. Since the work began on the project in 
January, 1966, the time span was two calendar years plus six months for 
preparing and publishing the revised manual and the final report on the 
project. Although the scope covered the 10 schools, some resources were 
shared with an additional school that had requested help in a local eval- 
uation. The personnel chiefly involved were the 20 project team members 

and the R & D project staff. 
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METHODS 



The Advisory Groups 



Two advisory groups were utilized in the initiatory stages of the 
project. One consisted of members of the vocational teacher education 
staff of MSU. They contributed advice on the design of the project and 



on the selection of schools. The state director of vocational education 
and his assistant also were consulted. The director appointed a staff 
member from each of the vocational education fields and one from the 
Michigan Research Coordinating Unit to serve as consultants. Two meetings 
were held, with these consultants serving as an advisory committee: one 



at the beginning of the project, and one near the end of the academic 
year 1966-67. They were invited to attend all meetings called for local 

leadership teams. They made themselves available to the schools as con- 
sultants. 



Selection of Schools 

In the selection of schools, the position was taken that all schools 
in Michigan should be considered. Thus, all schools were notified of the 
availability of the manual on evaluation previously referred to. Requests 
the manual came from about 250 schools* The five criteria used to 
select the schools from this group were: 

Have expressed a desire to evaluate its program with a view to improving 
it where necessary. The criterion was met if the school requested the 
manual of evaluation of local programs* 

--Have not less than 1500, nor more than 20,000 enrolled in kindergarten 
through 12 grade. 

Have vocational education programs under way in at least three reim- 
bursable fields. 

--Come from a location such as to contribute to geographical represen- 
tativeness of the group of schools as a whole. 
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— Have administrators and faculty willing to cooperate in the project 
by assigning a staff member as local project director or leader, and 
by providing him working time and local travel to direct the project. 

The size classification was made so as to have a somewhat comparable 
number in each class from which to select. These were small, 1500 to 
2,900 K-12; medium 3000 - 10,000 K-12; and large, 10,000 - 20,000 K-12. 

One school larger than this was included because it had already been 
selected as an R & D clinical school site, and because of its close prox- 
imity. On the basis of the criteria, letters of invitation to participate 
were sent to 25 schools. Of these, 16 responded favorably. Eleven schools 
were visited to provide the administrators further information. Invita- 
tions were sent to 14 schools to attend an orientation meeting on March 21, 
1966. Twelve of these sent representatives, and of these, 10 schools 
decided to become a part of the project. These included three large 
schools, three medium schools and four small schools as shown in Table Is 

Table I 

Size of Participating Schools in 1966 



Size 

Classification 


K-12 

Enrollment 


Name of 
School 


Large 


32,109 


La ns ing 




16,755 


Waterford Township 




11,900 


Benton Harbor 


Medium 


8,340 


Alpena 




6,510 


Niles 




4,700 


Sault Ste. Marie 


Small 


2,675 


Hillsdale 




2,600 


Fremont 




2,460 


Corunna 




2,043 


Big Rapids 



Each of these ten schools had reimbursable vocational education pro- 
grams underway in at least three fields. A diversified cooperative educa- 
tion program was treated as a field, since it denoted one or more fields 



in addition to those separately reimbursable. Business education is 
treated here as a field, including either or both office education and 
distributive education. Two of the 10 schools had programs in five 
fields, as shown in Table 2. The schools are scattered throughout the 
lower peninsula of Michigan, and one school is located in the eastern 
part of the northern peninsula as shown in figure 1. 

Table II 

Vocational Education Programs in Ten Participating Schools 



School 

Alpena 
Benton Harbor 
Big Rapids 
Corunna 
Fremont 
Hillsdale 
Lansing 
Niles 

Sault Ste. Marie 
Waterford Twp. 



Agricultural 
Education 

X 
X 
X 
X 
X 
X 



X 

X 



Business 

Education 

X 
X 



X 

X 



Home 
Economics 

X 
X 
X 
X 
X 
X 
X 
X 
X 
X 



Industrial 

Education 

X 
X 



X 

X 



Cooperative 
Education 

X 
X 
X 
X 
X 
X 
X 
X 
X 
X 



Leader ship Development Activities 



Hie activities of the project leader in the development of evaluation 
capability of local leadership teams were of five kinds: orientation to 

the project; workshop for leadership teams; periodic conferences or 
report meetings; development and dissemination of materials relating to 
evaluation; and consultant services. The orientation meeting was held 
for administrators and local leadership teams. It dealt with the following 
subjects: relation to the 8 & D program, purposes of the project, getting 

local projects under way, brief reports on each school situation repre- 
sented, and planning for future activities. 

A workshop of three and one-half days was held for the local project 



leaders and the research associates from each of the ten schools. Program 
participants included three persons from schools involved in the previous 
evaluation project; two staff members from the Michigan Department of 
Education, and 11 faculty members of Michigan State University. Attendance 
included 18 from the participating schools and three consultants from the 
Michigan Department of Education. Following the workshop memoranda of 
agreement with each school were drawn up and signed by the cooperating 
schools. (See Appendix A) 

The first report meeting was held on October 27 at MSU. Attendance 
included 17 persons from the 10 schools, two M.D.E. consultants and a 
representative of an associated school. The purposes of the meeting were 




A Progress Report Meeting of the Evaluation Systems Project, held at MSU, 
October 27, 1966, was attended by (from left to right): Langsford Collins, 

Sault Ste. Marie; Dr. George Kohrman, Western Michigan University, • consul- 
tant; John Ehresman, Alpena; William Davis, Niles; Richard Clark and 
Gerald Butts, Corunna; Edward Remick, Lansing; Dale Messerschmidt, assis- 
tant, MSU; Earl Miller, consultant, MDE; Arthur Nelson and Arnold Mokma, 
Big Rapids; Harold Byram, MSU project leader; Floyd McKinney and Joan 
Quilling, assistants, MSU; Walter Martinen and Arvid Erholtz, Hillsdale; 

T. H. Kerrey, consultant, MDE; Warren Suchovsky, Stephenson; Neal Blinkman 
and Donald Pobuda, Benton Harbor; Larry Wagner and Burton Thorn, Waterford 
Tow^'T-iip; and Fay Adair and Alvin Griffin, Fremont. 







The second meeting was held January 20, 1967. This was a short meeting 
scheduled in connection with a gathering of administrators and teams from 
all schools involved in any R & D projects. It was announced that R & D 
support to local schools was unavailable for the last half of the academic 
year due to cut in Federal funds. Nevertheless, all schools in the proj- 
ect decided to continue without the support- A brief report of progress 
was made by the project leader. Plans were made for future meetings. An 
inventory of information that local leaders might include in project re- 
ports was presented and discussed. All schools were represented by 15 
local leaders. 



An all-day meeting held on March 14 was attended by 17 persons from 
the 10 schools, an associated school, and three MDE consultants. The 
feature of this meeting was three group meetings according to size of 
schools, chaired by three research assistants. These groups later reported 
to the conference. Recent developments in the state project were reported, 
plans for completing of the project were discussed, and plans were made 
for year-end, depth interviews, and for the October implementation confer- 
ence. 



The sixth and final meeting of the project leadership teams was held 
on October 26, 1967. Twenty representatives from 10 schools, including 
two superintendents, representatives from three interested schools, 21 




Dr. Gordon McCloskey, 
noted specialist in 
educational communica- 
tions, Washington State 
University, addresses a 
joint luncheon of local 
leadership teams in the 
Evaluation Systems 
Project and state and 
local leaders in the 
Office Block Project. 

The subject of the 
discussion is "The 
Implementation Process." 
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staff members from MSU, and five MDE consultants attended, for a total of 
37. Reports by the project leader and ’'capsule" presentations by each 
school were featured. The conference divided into groups to discuss imple- 
mentation with leaders of four other R & D projects in vocational-technical 



education. Problems of implementation were also discussed 
meeting. The luncheon program featured an address by Dr, 
on "The Implementation Process," 



in the general 
Gordon McCloskey 



Materials developed and made available to leaders at sometime during . 
the two years included the Manual Evaluation of Local Vocational Education 
Programs ; papers presented by speakers at the workshop and the final 
meeting; the report of the previous evaluation project; a brochure de- 
scribing the project; suggestions on interviewing; suggestions on pre- 
paring the final report on the project; a guide for evaluating work with 
citizens' advisory committees; and an instrument for measuring understandings 
about vocational education. 



One of the major local leadership development activities was consultant 
services to the 10 cooperating schools. Limited consultant service also 
was extended to another small school at its request. The project leader 
provided much of this consultant service. Hie 10 schools received 54 
visits by him, with a range per school from three to 12. The number of 
these conferences varied somewhat indirectly with the distance from East 
Lansing. These visits were made to orient local leaders, to monitor the 
project, and/or to provide consultant service on problems and concerns. 

They included participation in four local staff and/or advisory committee 
meetings. The length of these monitor- consultant visits varied, but 
averaged about two hours. Thus, roughly 55 man-days were spent in this 
type of service, exclusive of time spent in reporting visits and preparing 
for them. A report of each visit or conference was prepared, with a copy 
)ing to each leadership team. 

The consultant services provided through research assistants were 
designated by field. More of the requests for their services, however, 
were not related to a particular field, but to research procedures and/or 
data proceeding. Seven were of this nature out of 24 requests. The 
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others dealt, in order, with industrial education, business and distri- 
butive education, miscellaneous, agriculture, and home economics. While 
no effort was made to determine the extent of State Department consultant 
services to these schools, six such cases were reported, all relating to 
a vocational education field. MSU staff other than project staff, served 
in a consultant capacity on a number of occasions, and made a total of 
eight addresses at meetings other than the workshop, previously reported 
here. 

Records. Reports and Interviews 

Records kept included a log of activities kept by the project leader; 
materials submitted by the cooperating school; reports of visits/confer- 
ences with schools; correspondence with them and monthly reports to the 
R & D Program Director. Each school was asked to prepare and submit a 
final report on its project. Nine such reports were received. Each 
leadership team kept a record of surveys or other research activities, and 
results and reports of investigations. Each local leadership team kept 
a log of its own activities. 

The structure of depth interviews conducted at the end of the academic 
year was developed by the project leader, with the help of the research 
assistants and the MDE consultants assigned to the project. The interview 
forms were tried out in one school before start of these interviews. The 
interview forms are given in Appendix B. 

Public/professional relations on the project were designed to inform 
the local schools and committees, the education profession, and the gen- 
eral public. Presentations in regard to the project were made to a state 
RCU conference, to a RCU-R & D joint meeting, to a MSU Department of Sec- 
ondary Education and Curriculum meeting, to a U.S.O.E. Vocational Division 
meetings five times to a University of Michigan Seminar for prospective 
administrative leaders, once to a conference of R 6c D Hospitality Educa- 
tion Project participants, and six times to MSU graduate classes in 
vocational education. A brochure was produced and received wide dissemina 
tion. Two newspaper releases were widely published. Each participating 
school conducted local publicity regarding the project. 
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Two special studies were planned and conducted by research assistants 
assigned to the project. One related to the role of the consultant on 
vocational education evaluation. This is raferred to in Objective No. 4 
of the project. The other was the development and preliminary tryout of 
an instrument designed to measure understandings about vocational educa- 
tion in the public schools. (See Appendices E and F for complete reports 
of these studies) 
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II. FINDINGS and ANALYSIS 



This section is mainly a presentation of plans made and activities 
conducted by local project leaders, the faculty in their schools, and the 
citizens in their communities. The focus is on the year 1966-67, although 
many of the outcomes of the projects would not develop or be identifiable 
until later. 



LOCAL PLANS FOR EVALUATION 



All leadership teams prepared plans for conducting their evaluation 
projects, but there was little uniformity in the contents or format of 
these. Five prepared statements of objectives for the project. Two 
presented a statement of overall, or general purpose. Three did not give 
the purpose or objectives in the written plan. There had been no require- 
ment placed on these schools in this regard, but it had been suggested at 
the start of the project by the state project leader that objectives 
could well be included in the plan. There was evidence that in some 
cases the local leadership team wanted the staff evaluation committee to 
formulate its own objectives. 

Diversity among the local project plans also showed up in the listing 
of proposed activities. Two of the large schools included a long list -- 
in one case of 17 activities and in another case of 14 — for the com- 
mittees and staff to consider and/or carry out. The other large school 
presented: (1) an outline for each departmental self-evaluation; 

(2) the plan for the evaluation committee; and (3) the plan for advisory 
committee use. One of the medium schools listed specific activities in 
its plan, but the other two did not. Among the small schools, one listed 
specific activities, one suggested areas of activity, one presented the 
plan for staff involvement and things to be studied, and one mentioned 
one activity which was to be its main endeavor. 
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Four of the schools included a complete schedule in their plans, and 
two presented partial schedules. As with the question of objectives and 
activities, several leadership teams preferred to encourage the develop- 
ment of these through staff and/or community involvement. 

Most schools did not include a plan for use of consultants. The 
three that did so app-ared to mention it as a reminder to themselves or 
to the educators with whom they worked locally. 

The administrative relations to the local projects varied, as would 
be expected, with size of school and nature of staff and administrative 
organization. These will be reported by school size, starting with the 
large schools. 

One of these organized a "central committee" that included the assis- 
tant superintendent and the high school principal. Another one had a 
"Vocational Education Study Committee Organization" composed of adminis- 
trators from each of the junior and senior high schools. This committee 
met with the vocational and practical arts teachers and the assistant 
principal in each of the two high schools, and some junior high school 
teachers at the end of the preceding school year. The third large 
school had an administrative committee consisting of the superintendent, 
his deputy superintendent, the supervisor of instruction, the director 
of secondary education, and the principals and assistant principals of 
the three senior high schools. The leadership team met weekly on the 
project and monthly with the state project leader. 

The three medium schools varied in their administrative relations. 

In one the superintendent assigned the responsibility for leadership in 
the project to the team and gave very little attention or supervision to 
it. In another school the superintendent appointed the leadership team 
and called a meeting to start off the local evaluation committee. Included 
in this, in addition to teachers, were the high school principal and 
assistant principal and the junior high school principal. The superin- 
tendent in the third school recommended appointment of the leadership 
team. The assistant principal of the high school and the director of 
guidance was placed on the steering committee for the project. The 
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superintendent kept in constant touch with the leadership team in this 
school. It had the greatest amount of citizen involvement of all ten 
schools. 

In the four small schools, all projects were inaugurated by the super- 
intendents. In one of these the assistant superintendent was designated 
as a project leader. In another this position was represented on the 
evaluation committee. In a third school no plan for communication through 
the high school principal was set up. This apparent oversight was later 
referred to by the superintendent as a weakness in the project. In this 
school a meeting was held before the end of the previous school year to 
launch the project. In the fourth school there was a change in the super- 
intendency between the time of decision to cooperate in the project and 
the launching of the local project. This change may have adversely affected 
the momentum generated at the outset. 



LOCAL STAFF INVOLVEMENT IN THE PROJECT 



Numbers and Proportion 



The presentation and analysis of information about staff involvement 
focuses first on the scope, or extent of involvement. The ten cooperating 
schools reported a total of 297 staff members involved, in addition to the 
20 leaders, or a grand total of 317 local educators. In terms of the num- 
ber in each school, there was considerable variation. The three large 
schools involved 20, 47, and 90, for a total of 157. The three medium 
schools involved 13, 30, and 49, for a total of 92. The four small schools 
had 6, 11, 15, and 16, for a total of 48. 



Analysis of fields of staff positions showed that most were vocational 
and/or practical arts teachers. The largest number, 84, were in office 
and/or distributive education, followed by 75 in industrial education. 
These would include industrial arts teachers in the smaller schools as 
well as in some of the larger schools, and health occupations teachers. 
Both these groups also included coordinators of cooperative occupational 
education. Home economics teachers numbered 19 from nine schools, and 

agriculture teachers eight, one for each of the schools offering such pro- 
grams . 



Seven schools reported the involvement of 17 members of guidance 
staffs, in addition to the three schools in which the research associate 
was drawn from the guidance staff. Two large systems did not involve 
these persons. The non-involvement of home economics staff members also 
was in one large system. This may be a reflection of the local definition 
of what is included in vocational education and/or the jurisdictional 

lines drawn in regard to the local director of vocational education serving 
as project leader. 

Other than the faculty groups separately identified here, there was 
a total of 81 persons. This additional involvement ranged from one to 
25, but this number seemed unrelated to size of total staff involvement. 



These persons included administrators and/or their assistants, and academic 
teachers. Two teachers or administrators of adult education were included. 
The project leader who conducted a credit workshop involved 30 through 
this medium, in addition to 17 others. 

The generally prevalent, non- involvement of persons identified with 
adult education can be ascribed to several factors. Generally, the adult 
teachers were not full time in this assignment. Some schools administered 
adult programs through the local community college, again the jurisdic- 
tional lines showing up. The result, however generally speaking, was the 
restriction or concentration of evaluation in most schools to the secon- 
dary level. 

Extent and Nature of Involvement 



The number of meetings held by local leadership teams might give 
some indication of the extent of involvement. However, not all schools 
reported this information. The five that did, showed a range from one 
to seven, for a total of 15, and average attendance ranging from 4 to 13. 
But in addition, these and other schools involved staff members in citizen 
advisory committee meetings, which are reported in another section. It 
should be noted also that all project leaders worked directly with indivi- 
dual teachers, small teacher sub- commit tees and in other informal ways. 

The project leader in one school who conducted a credit workshop held 
eight three-hour meetings of this group. 

Factors enhancing involvement were identified as competence of the 
leader in group processes, administrative endorsement and encouragement 
to faculty members, and arrangements for time to work on che evaluation. 
Inhibiting factors differed by schools. In one school the high school 
principalship was vacant during the year and was a controversial community 
subject. In one school the research associate who had been prepared to 
function sought and obtained another assignment, and the staff member 
replacing him had to learn his role. In another school a new superinten- 
dent was in charge and "feeling his way", while in still another school 
the superintendent left for a new position before the project year ended. 
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In three schools the project leader was not in an administrative or super- 
visory position, which might not have been a handicap, but was reported 
as such by these leaders, particularly as related to the enlistment of 
staff cooperation and to implementation of recommendations. 

There were some differences with respect to the start of staff involve 
ment. Two small schools started in May. One large school started pre- 
liminary steering committee work in June, All the others started just 
prior to, at, or soon after the opening of the school year. 

Methods Used 



The form of involvement most common was the organizing of a staff 
steering committee, or committee of vocational department heads. This 
was the pattern in two large, three small, and one medium school. The 
forming of an evaluation staff and/or the grouping of all the vocational 
and practical arts teachers was followed in two medium and one small 
school. An administrative announcement and individual contacts were used 
in the remaining school. 

General Outcomes 



The outcomes from staff involvement in the cooperating schools could 
not be isolated or identified separately from the outcomes of the project 
as a whole. In the year-end interviews the leaders and their associates 
were asked about the sufficiency of involvement. Those from seven schools 
reported sufficient involvement and three insufficient. Of eight admin- 
istrators interviewed, five reported adequate communications from those 
involved and two inadequate. One said the oral communications were 
better than written ones. leaders from eight schools volunteered the 
belief or perception that the philosophy of vocational education held by 
staff members had changed. One leader early in the project reported 
that the first meeting was one which, to his knowledge, was the first 
time that all the vocational teachers had ever met together as a group 
and talked with one another! 



The final report by one of the large schools gives additional insight 
into factors enhancing involvement in that school. 

’’Involvement of faculty members, counselors, and administrative 
staff was the single most difficult item to obtain in evaluating 
the vocational education program. Because these people were con- 
cerned with and involved with multiple daily procedural details, 
they were reluctant to assume additional responsibilities. They 
would, however, accept additional tasks when these were of some 
personal interest.” 

’’One successful approach to having certain evaluative activities 
performed was to engage those faculty members, counselors, adminis- 
trators, or others who were enrolled in graduate programs. Fre- 
quently these persons would complete the project in conjunc ion with 
a course. Advantages of these techniques were: 

1. The project had a deadline date for completion. 

2. Tighter controls were placed on data sources and data 
gathering. 

3. Reference materials were checked for comparisons of response. 

4. Classmates sometimes showed interest and provided suggestions 
and encouragement.” 

’’Practices found to be unsuccessful in gaining participation 
included : 

1. Assigning a staff member a specific problem to be completed. 

2. Calling general staff meetings to request participation.” 



- 30 - 



o 



CITIZEN INVOLVEMENT 



Advisory Committees. Members, Size 

The involvement of local citizens in evaluation will be recalled as 
an important part of the project design and the local schools participa- 
tion in the Evaluation Systems Project. Six of the 10 schools associated 
with the project used citizens’ vocational advisory committees as part 
of their evaluation effort. These six schools represented the three 
size classifications of the schools in the project. Two of the schools 
were large, one medium, and three small, according to the size classifi- 
cation adopted for the project. 

The six schools involved a total of 45 different advisory committees. 
The smaller schools tended to have committees organized by the following 
fields: home economics, agriculture, business and distributive education, 

woodworking, and industrial education. Usually the larger the school 
the more extensive the vocational offering, and therefore the larger the 
number of citizen committees. Other committees used in the larger schools 
included: auto-mechanics, electronics (electricity), body shop, residen- 

tial drafting, machine drafting, drafting, printing, graphics, machine 
shop, metals, engine repair, job advisory, health occupations, food ser- 
vice, air conditioning, building trades, trade and industry, business 
and distributive. 

These 45 committees involved a total of 303 citizens, for an average 
of 50.5 citizens per school and an average of 6.7 members per committee. 
The citizens* committees in five schools were established as a result of 
the involvement of these schools in the Evaluation Systems Project. One 
of the schools had at one time established and used citizens' committees. 
There was considerable variation in the number of meetings held by the 
committees. In some schools the committees had normally met once a year, 
while in others the average had been four to five meetings per year. 

Newly established committees tended to meet more frequently. 
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Nature of Involvement 

One or more of the schools reported each of the following methods as 
having proved successful in the operation of their advisory committees. 

--Vocational departments nominated citizens for committees. 

--Meetings held at the convenience of non-educators. 

--Informal meetings were more effective than formal meetings. 

--Use of publicity to inform the community of what is happening. 

—Selection of committee chairman usually a committee decision. (In 
some schools a faculty member served as chairman in all cases, while 
in at least one school a faculty member was chairman only if there 
was lack of leadership on the part of the citizen members of the 
committee. ) 

--Committee membership including representatives of organized groups, 
employers, supervisory personnel, and influential citizens. 

--Use of personal contacts, visits, and phone calls to secure citizen 
participation in the committee activities. 

--Committee members kept well informed on the school s program. 

--Meetings held at night with agenda prepared in advance by the committee 
chairman in cooperation with school liaison personnel. 

--Refreshments served at committee meetings, courtesy of the school. 

It can be noted that a practice effective in one school may prove to 
be ineffective or less effective in another. Not all of the methods 
just listed were found to have been successful in a single school. It 
was necessary for each school to adapt procedures to its own individual 
needs and problems. In some schools the committees operated as craft or 
vocational field committees, while in others there were field or depart- 
mental committees which were brought together as a general vocational 
citizens' advisory committee. 

The amount of leadership provided by the local project leader and 
research associate varied greatly among the schools. In one school in 
which the citizen involvement was very successful, in terms of meetings 
held and citizen attendance and involvement, the project leader and/or 
research associate attended every committee meeting. Citizen members of 
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the committee were contacted by phone prior to the meeting by the project 
leader's secretary. 

Involvement of the citizens included the following types of reported 
activities . 

--Familiarized members with present facilities and program offerings. 

--Assisted in the preparation of follow-up questionnaire forms. 

— Reviewed and discussed findings of follow-up studies and staff 
visits to places of employment. 

--Formulated recommendations. 

— Developed new ideas (brainstorming sessions). 

— Discussed and made recommendations relative to course and program 
objectives and new courses. 

--Made specific suggestions in regard to activities in a building 
trades class. 

Unreported activities probably included discussion of objectives of 
urograms. Evidence of this is that generally, the schools using advisory 
committees reported an increase in the interest on the part of citizens 
in the vocational programs. Committee members were also reported as being 
helpful in informing school personnel about the training needs of the 
community. 



FOLLOW-UP STUDIES AND INTERVIEWS 



Kind and Number Conducted 



The staff evaluation committee in each school considered the making 
of follow-up studies when planning activities appropriate to attain the 
objectives of its project. Three small schools, one medium school, and 
one large school, conducted follow-up of all graduates of a given year. 
Two medium schools and one large school had recently conducted general 
follow-up surveys, and the committees in these decided to devote time 
and energy to other activities. One large school had previously had a 
number of such surveys made, but also conducted several special studies 
that will be referred to later. One small school followed up its former 
students of agriculture. 

Survey of All Former Students 

In several respects, the most successful survey was one conducted by 
a small school. The graduates of 1961 and 1965, numbering 212, returned 
167, or 79 percent of the questionnaires sent them. These were coded 
and processed at the MSU computer center. All recommended pracj^cjes 
regarding procedures were followed by the staff committee. All‘ ! 'doiinmittee 

i 

members participated in the development of the instrument and in the 
interpretation of the results. One finding came as a surprise to most 
people involved, namely a much higher proportion of former students hav- 
ing taken employment within a 35 mile radius and remaining in it. This 
has led to a more detailed study of businesses and industries in the 
immediate area, and a strengthening of the vocational guidance program 
in this school. Plans were started to make the follow-up a continuing 
one, and an integral part of a placement service subsequently to be 
organized. Curricula are bing re-studied in the light of the findings 
and some programs are being expanded. 
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A staff committee of a medium school conducted a follow-up survey 
of the 1966 graduating class of 401 alumni. Information was obtained on 
all former students in this category. In addition to a mail questionnaire 
from 175 former students, 200 telephone calls were made to secure infor- 
mation. This was supplemented by newspaper clippings, grade reports from 
colleges, and information from in-school students and teachers. While 
some questions were asked of all graduates, a different form was developed 
and used in each vocational field or department. This was sent to former 
students under a cover letter written by a vocational teacher. Curricular 
suggestions, and a basis for a continuing follow-up system and placement 
service were reported as outcomes by the committee. A part of the latter 

came out of another survey conducted by counselors of the intentions of 
the 1967 graduating class. 

One of the small schools studied the graduates and other former stu- 
dents who were one, two, and five years out of high school. Separate 
objectives were listed for each year's probe. Again, the staff committee 
participated in development of the survey instrument. All other proce- 
dures were followed as those in the school just reported. However, re- 
turns were received from only 55 percent of the 505 graduates and 40 
other former students. The two later graduating classes returned 61.6 
and 64.1 percent, respectively. Not unexpectedly, the drop-outs returned 
only 12.5 percent. The findings from the survey supported local impres- 
sions, namely, that former students were finding employment in the area, 
and that local occupational information should be considered more seriously. 
The graduates' evaluation of the curriculum was not used. This was because 
of a weakness in the check list which was a provision for the respondents 
to rate courses rather than competencies acquired or activities engaged 
m. The returns were such as to strongly suggest that the respondents 
had rated former teachers, rather than the courses they took. Also, many 
respondents did not rate courses they had taken. Another weakness was 
that of asking for the respondent's name on the questionnaire. This 
school has since decided, that in future surveys it wills code its ques- 
tionnaires; limit the survey to employment -bound graduates if the guidance 
department continues to follow-up the college-bound students; re-struc- 



-36- 



f 



tare the curriculum evaluation section; and clarify some questions 
that appeared to be ambiguous. 



Another small school had a similar experience, with only a 45 percent 

return. Weaknesses in procedures responsible for the lower return were 

similar to the one just reported. A second factor was a past history of 
less than cl 

1 ‘ ln ruture studies this school plans 

to exploit the alumni relationships, and to re-structure the curriculum 
evaluation section. 



The large school that attempted a general follow-up included former 
students of agriculture, distributive education and other cooperative 
occupational education programs, and of machine shop. The instrument 
was simple, one-page form dealing mostly with employment status. While the 
brevity of the form was expected to contribute to a response, a number 
of factors have been identified as weaknesses, inferior procedures, or 
handicaps that accounted for the very low return and the consequent lack 
of value of the survey. The form went out under a cover letter, not 
from the principal or other administrator, but from the project leader 
who was not well known to most former students. The effort to enlist the 
support and cooperation of the guidance staff in this activity had not 
succeeded. The form had some ambiguous questions, and was not presented 
m such a way as to appeal to respondents. There had been no previous 
preparation of graduates for the survey. The survey form was sent to 
former students too soon after graduation — less than a year — and with- 
out any indication having been given to them the previous year that they 
would be contacted for information. 



Special Follow-up Surveys 



The large system in which follow-up surveys previously had been made 
did engage in several follow-up activities. One of these was to field 
test a survey instrument. A qualified interviewer was hired to interview 
students enrolled in vocational education programs to help establish para- 
meters of response to questions to be used. There appeared to be evidence 
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of gaps in knowledge which students have about the world of work, repre- 
sented by the courses in which they were enrolled. A survey of graduates 
of the day- trade program of 1966 was made. Responses showed none of the 
respondents was unemployed. All were using some of the skills and know- 
ledge learned in the courses. A third follow-up activity dealt with the 
former students of one of three high schools. This was a part of a 10- 
year longitudinal study in its fifth year. A random sample of one half 
of the 1962, 1963, and 1966 graduates were surveyed. All drop-outs also 
were surveyed. Although the percentage response from the total group was 
disappointing, the 1966 class produced a 59 percent return. The major 
objective was to obtain information regarding the effectiveness of the 
curriculum and related factors. Information was secured on employment 
and training needs, skills learned, and value of high school experiences 
to them in subsequent employment situations. As a result the surveyor 
had questioned the wisdom of seeking information from persons several 
years out of school regarding curricula because of rapid change in them. 

In the other large system, surveys had been made in previous years, 
so the staff evaluation committee called in consultants to help it evalu- 
ate its follow-up procedures. As a result, it has been decided to enlist 
student help in developing questions, to try out the form with students 
yet in school, and in other ways to motivate and condition students to 
all cooperate in responding to a follow-up form, after leaving school. 

The staff in office education also developed a form to be used in the 
near future to discover employment information about former business 
students. A third survey was made, this one going to senior students to 
obtain information for guidance, specifically in regard to plans for 
employment and/or further education. 

Another survey in one of the small schools was made to follow up 
former students of agriculture. This was planned by the advisory com- 
mittee for vocational agriculture, and included graduates of the pre- 
vious five years who had taken four years of vocational agriculture. All 
but one of the 81 graduates in these years were accounted for. Those 
employed in agricultural occupations, or preparing for them, accounted 
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for 35 percent of the total, and 45.2 percent of those not going into 
military service* 

Generalizations Regarding Successful Practices 



Several generalizations can be made regarding follow-up studies, as 
part of local program evaluations. These are tentative, are subject to 
further trial, grow out of the previous evaluation project, and/or reflect, 

in part, literature on the subject. 

--The purposes of the survey should be clearly stated and adhered to 

in all steps. 

— The instrumentation and data gathering techriiques should be those 
that are most effective in eliciting reliable and valid information. 

--Administrative endorsement should be obtained to provide status to 
the study, support for financing, staff time, and implementation of 

the findings. 

--The greater the staff involvement and participation in planning, the 
more effective will the study be. 

--Students should be involved in development and try-out of instruments, 
and should be motivated before graduation to cooperate in follow-up 
studies. 

— Citizens’ advisory committees can help in setting objectives, in 
promoting cooperation by the people, in interpreting the findings, 
and in communication to the community. 

--Surveys should not be made in the same calendar year as the one in 
which the student graduated, and normally not later than five years 
after graduation. 

--Evaluations of competencies developed and of competencies needed on 
the job should be solicited, rather than evaluation of courses. 

--Evaluations of competencies acquired by former students should be 
supplemented by related information from employers. 

--The instrument used for gathering data should be constructed by a 
person with competence in research techniques. 

--Tested mailing procedures should be followed. 

--If large numbers are involved, and the instrument is long, electronic 
data processing procedures should be used. 
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should be estimated and provided for before 



—The cost of the survey 

the study is launched. 

—The use of the telephone to supplement mailing generally adds to 

efficiency of the survey. 

The reactions from those schools in which the follow-up studies were 
deemed successful show an intention to repeat or make permanent the prac- 
tice of conducting follow-up studies in future evaluations. The schools 
that were unsuccessful in this activity, nonetheless, intend to try it 
again and to correct their mistakes. The schools that had previously 
conducted a follow-up survey indicated their intentions to conduct such 
studies in the future. This leads to the genet alizat ion that such studies 
should become a universal activity in projects in all schools conducting 
evaluations of local programs of vocational education. 



STUDY OF NEEDS -- CURRICULAR ANALYSIS 



"No longer can the emphasis be on matching the best man with 
an existing job; it must be placed on providing a suitable job 
for each man or equipping each man to fill a suitable job... 

"Career consciousness must be integrated throughout the 
school s. M 8 

This quotation from the report of the National Advisory Council for 
Evaluation of Vocational Education epitomizes the point of view of the 
schools involved in the project. One of the suggestions originally 
given to them was that occupational training needs be studied and school 
curricula examined in the light of the projected needs. One way to 
implement this suggestion would have been to conduct systematic, or for- 
mal occupational surveys in each community. This was not done for several 
reasons. At best, the survey would have been limited to the geographic 
area served by the school. With one exception, namely the largest school, 
this geographic area was smaller than the area in which students would 
later become employed. The available data on occupations in the state 
were much too gross to be of great practical use in the small and medium 
schools. 

Study of Documents and Recorded Data 

Michigan was being covered by a series of area vocational education 
studies — surveys to determine the needs for area vocational- technical 
education programs. These had been completed or nearly completed in the 
areas in which all but one of the schools were located. The reports of 
these surveys were available and had been studied by these nine schools. 
The relatively high sophistication of these schools with respect to occu- 
pational opportunities, trends, and training needs was to be expected. 

This is because the schools in the project were to a considerable degree 

8 " ” 

Advisory Council of Vocational Education, Vocational Education: The 

Bridge Between Man and His Work , Summary and Recommendations, (Washington, 
D 0 C. : U.S. Government Printing Office, 1968), p. 4. 
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self-selected. They had been categorized by state leaders as schools 
that were aware of needs, and making significant strides toward meeting 



these in school programs. The tenth school became involved in the area 
study during the year of the project. Some of the activities, and devel- 
opment of recommendations in this school were necessarily postponed 
because of this. The leaders desired to cooperate in the area study, and 
believed it would supply needed information for evaluation and planning 
purposes. 



Interviews and Questionnaires to Employers 

The information from these sources was supplemented by project leaders 
in nine schools through informal questionnaires and interviews of key 
individuals in business and industry, as well as through solicitation of 
information and suggestions from advisory committee members. In no 
case, however, was the information obtained in sufficient detail for occu- 
pations covering a wide range of fields, so that curricula could be com- 
pared directly with the specifics of occupational requirements. On the 
other hand, considerable specific information was obtained in this way 
for analysis of office education courses in one large school, one medium 
school, and two small ones; of certain phases of industrial education 
in one large, one medium and two small schools; of distributive education 
in one large, one medium, and two small schools; and of others. 

Some of these analyses were made through interviews and/or question- 
naires by staff or students. For example, in one of the small schools 
a questionnaire was sent to 42 employers of office workers in the area. 
Responses from 29 of these provided helpful information regarding degree 
of competencies possessed by newly-hired employees and competencies 
employers look for in office workers. In this same school a business 
class interviewed 32 office secretaries and obtained much valuable in- 
formation for curricula improvement. Another small school conducted 27 
employer interviews in the community to learn first hand of employment 
situations and job training needs. 




Other analyses were made through citizens' advisory committee discus- 
sions. The reports of the deliberations of these committees, in the seven 
schools that used them, are replete with information on occupational train- 
ing needs and with statements relative to the reality or viability of pro- 
grams and curricula. The outcomes and procedures followed with these 
committees are found in a preceding section of this report. 



SPECIAL LOCAL ACTIVITIES 



Credit Workshop 

As a part of its evaluation effort one of the schools offered a uni- 
versity credit workshop on evaluation of practical arts and vocational 
education. The workshop enrollment was limited to staff personnel of the 
local school, and included administrators, high school academic teachers, 
and elementary teachers, as well as vocational and practical arts teachers. 

The local vocational director, who had previously been a faculty member 
at Michigan State University, served as the workshop instructor. The nine 
workshop sessions served as the core of the school's evaluation study. 
Workshop participants formed problem-area groups which fostered involve- 
ment at greater depth into specific interest areas related to the total 
evaluation effort. 

Activities growing out of the workshop included an attempt to estab- 
lish a placement service for the school's youth; improved orientation of 
youth toward work and career opportunities; a follow-up study of business 
education graduates; a study of youth employment in the senior high 
schools; a plan for longer blocks of time for certain industrial arts 
courses; and evaluation of the health occupations cooperative education 
program, including related instruction; a study of attitudes of teachers 
toward vocational education; a junior-high program of practical arts; 

and a program of unified arts and occupational education in the elementary 
school. 

The school personnel considered the workshop to be very successful. 

The workshop, in the strict sense, did not limit the participants* acti- 
vities to those purely evaluative in nature. The workshop was perhaps 
most effective in bringing together a cross section of people, K-12, 
including teachers, administrators, and counselors for discussion and 
program planning for education for the world of work. 





Study of Attitudes Toward Work 



The staff members associated with one of the local school projects 
developed an instrument to measure attitudes concerning work and toward 
vocational education,. The instrument was administered to the elementary, 
junior "high and senior-high students, as well as to teachers and adminis- 
trators. The local staff member responsible for analyzing the data secured 
from the instrument resigned from his position without completing the 
summary of the study, therefore no information is available concerning 
the findings or validity and reliability of the instrument. 



Three-School Meeting 

Midway through the project year representatives of the three medium- 
size schools met at one of the schools to confer with each other in regard 
to common problems they might have related to their evaluation efforts, and 
to tour the facilities of the host school. Fourteen local school person- 
nel and a MSU Evaluation Systems Project Research Assistant attended the 
meeting. The local school personnel represented local project leaders and 
research associates, school administrators, counselors, and vocational 
teachers. 

Activities of the day-long meeting included a discussion of the evalu- 
ation project in the three schools, a discussion of future programs, and 
a tour of the host school facilities with emphasis on those for vocational 
education. 

While no formal evaluation of this meeting was attempted, all of the 
participants were most enthusiastic in their support of this type of 
activity. The fact that sister schools were engaged in various activities 
without doubt served as a stimulus for action in the schools. 



Catalog of Vocational Courses 



One large school, as part of its totai effort in the project, developed 
an informational booklet describing their educational offerings in occupa- 



tional development. The booklet included major emphases and outcomes, and 
typical activities for each subject taught in distributive education, hos- 
pitality services education, industrial arts education, office education, 
pre- employment program, and trade and industrial education. Eighty-three 
courses are listed in the booklet which serves as a useful resource to 
teachers and counselors for helping youth and adults plan their educa- 
tional programs. 



SPECIAL STUDIES 



Two special studies were made that relate to the project as a whole, 
rather than to any one cooperating school. One of these was conceived to 
contribute to attainment of Objective no. 4 of the project relating to 
the role of the consultant in evaluation of piograms of vocational educa- 
tion. The other was a ’’spin off" study made in response to a desire 
expressed by a number of local leaders to more effectively assess the 
understandings and/or attitudes of local faculty members regarding voca- 
tional education. The result was the development and try-out of an 
inventory. Complete reports of these special studies are included in 
Appendices E and F. In this section a brief review of each is presented. 

The Role of the Consultant in Evaluation of 

Local Vocational Education Programs 

Summary of the Study ; Two questionnaire forms were developed to ascer- 
tain the role of the vocational education evaluation consultant. One fcr 
consultants and one for consulters. Of the 49 questionnaires distributed 
to the consultants, 31 were returned, for an 83 percent return. Twenty 
of the 22 consulters receiving questionnaires returned the instrument, 
for a 91 percent return. 

Analysis of the data led to the following conclusions, which apply 
to the population of the study, in regard to the five questions for which 
the study sought information. 

1. What is a consultant? 

There was agreement between consultants and consulters that 
a consultant on vocational education evaluation is primarily a 
resource person. While the consultant as an expert was held second 
in importance by consultants, this was somewhat subordinated by 
consulters to the concept of the consultant as a process person. 

Over one-half of the consultants responding to the question- 
naire had spent less than 20 percent, however, had spent 80 to 
100 percent of their time serving in a consultative role. 
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Consultants had spent most of their time with administrators, 
with teachers following closely in second place. The least amount 
of time had been spent with other consultants and advisory commit- 
tees, but with local groups receiving somewhat more time. 

The planning stage is when most consultants 1 services had been 
used. Of less importance were the evaluating, implementing, and 
innovating stages. 

The consultants had spent most of their consulting time in 
the capacity of furnishing material and resources. The least 
amount of time had been spent as an expert on evaluation. Of less 
importance was assisting persons in solving their professional 
problems. 

2. Is there a difference in the way the consultant and consul ter 
perceive the role of the consultant in vocational education evalu- 
ation? 

The rank difference coefficient of correlation between consul- 
tant and consulter was relatively high for all areas of the study 
questionnaire. Highest agreement, .904, was related to the work 
of the consultant in evaluation and the lowest, .692, in regard 
to the consultant's abilities and characteristics. The degree 
of agreement of the consultant's traits and his activities was 
.825, and .832, respectively. 

3. What abilities and characteristics should the consultant possess? 

The respondents felt the abilities and characteristics of the 
consultant should be oriented toward planning, assisting, and 
recommending, rather than to decision making. 

Regarded as highly important were the abilities to plan, to 
convey clear thinking, to maintain a close working relationship 
with group leaders or directors, and to keep up to date on materials. 

The highly important characteristics were to be able to main- 
tain rapport within the group, to be democratic, to be able to 
accept people, and to grasp the scope of the situation. 
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4. What activities can the consultant perform to stimulate evaluation? 

There was agreement on the consultant's activities as including 
planning, suggesting, recommending, listening, criticizing, stim- 
ulating, organizing, clarifying, and synthesizing. 

The most frequent activities reported were collecting and 
recommending resources for use by people involved and listening 
to problems and plans of groups of participants. 

5. What guidelines may be used to direct the consultant in his work? 

Based on the data analyzed and review of related literature, 
the consultant should: 

— diligently prepare himself and furnish resource materials 
for the consul ter s 

— have a broad background of experience in his field before 
serving as a consultant 

— personally visit the area and the people connected with the 
problem or situation 

— be dynamic, practical, democratic, sympathetic, and a good 
organizer 

— assist the group to arrive at a solution to their problem 
by offering alternative solutions to it 

— assist in setting goals and in interpretating information 

— help the group move toward making a decision 

— help evaluate group goals, processes, and decisions 

Background of the Study : Experience of the staff associated with the 

project prompted an interest in attempting to determine the role of the 
consultant. This resulted in part from the experience of the project per- 
sonnel in the performance of consultative activities for the local schools 
associated with the project. These schools were also using consultants 
from other sources, especially those with the Michigan Department of 
Education. Most of the study was conducted by a project research assis- 
tant with the cooperation and assistance of the project director and 
other project research assistants. 
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Problem : Consultants for vocational education evaluation may not 

always be utilized effectively because they, or the people who use their 
services, do not have a clear concept of their role. This lack of know- 
ledge concerning the role of the consultant could result in a less capable 
performance that would be possible if the consultant and consulter were 
to better understand the situation. The problem then, becomes one of 
determining the role of the consultant in vocational education evaluation. 
This study attempted to answer the following questions: 

1. What is a consultant? 

2. Is there a difference in the way the consultant and consulter 
perceive the role of the consult ant in vocational education 
evaluation? 

3. What abilities and characteristics should the consultant possess? 

4. What activities can the consultant perform to stimulate evaluation? 

5. What guidelines may be used to direct the consultant in his work? 

A consultant was defined as a person outside the school staff and its 
clientele that might be affected directly by decisions; who has had expe- 
rience and/or education that qualify him to give advice, information, 
interpretations and/or suggestions regarding alternative courses of action 
or procedures, and/or who has skill in helping professional people to 
identify and work through their problems to a satisfactory solution. 

9 

Consultant types were recognized as those identified by Savage. These 
include the ’’expert,’* the ’’resource person," and the "consulter” — one 
who utilizes the services of the consultant. 

Procedures and Development of Checklist : To determine how the con- 

sultant and the consulter view the role of the consultant on evaluation 
in vocational education, two questionnaires were developed. ^ Question- 



William W. Savage, Consultative Services to Local School Systems , 

(Chicago: Midwest Administration Center, 1959). ^ ' 

The plan for the study and instruments were developed by Joan Quilling, 
who also gathered the data and made a preliminary analysis. The analysis 
was completed and the report prepared by Floyd McKinney. Both were re- 
search assistants on the project under the supervision of Harold M. Byram, 
project leader. 
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ng irs Form I was developed to determine how consul tants perceived their 
role and Form II was utilized to determine the expectations consulters 
held for the role of the consultant. Items for the questionnaires were 



developed after conducting a search of the literature on the role of the 
consultant. The questionnaires were then reviewed by the project director 
and research assistants. The questionnaires were revised and submitted 
to state leaders in vocational education xn Michigan. Using their sug- 
gestions, further revision was made and the forms were submitted to a 
jury of teacher educators and state consultants. After further revision 
the forms were submitted to, and approved by the U.S , Office of Education. 



The Populations : The population used for Form I included 31 leaders 

in vocational education in Michigan universities and in the Michigan 
Department of Education. Seventeen of the respondents were vocational 
teacher educators at Michigan State University, Wayne State University, 
Central Michigan University, Western Michigan University, Eastern Michi- 
gan University, and Northern Michigan University. The consultants from 
the Michigan Department of Education were all employed in the Vocational 
Education Division. 



Design ; The forms were mailed to respondents. The major portion ot 
the questionnaire was designed so that the respondent could easily check 
one of several responses to an item or check a response indicating his 
agreement or lack of agreement, the importance of the item, or the fre- 
quency with which the item should be done. 

The following steps constitute the general design of the study. 

1. Conduct a review of literature to determine items for the ques- 
tionnaires. 

2. Utilize the Evaluation Systems Project staff in an advisory 
capacity in developing the questionnaires. 

3. Have a jury of experts review the questionnaires. 

4. Utilize local project leaders and research associates connected 
with the Evaluation Systems Project and vocational educators in 
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Michigan Universities and the Michigan Department of Education to 
respond to the questionnaire. 

Analysis : Thirty-one individuals responded to Form I of the question- 

naire. Assuming the respondents would equally divide their choices, the 
items with three-response choices would have about ten responses per 
choice, and the items with four-response choices would have about eight 
responses per choice. If a rating of 2-1-0 is assigned to the items with 
three-response choices and a rating of 3-2-1-0 to the items with four- 
response choices the total score for such items ^ould then be 30 to 48 
respectively. For an item to be rejected it was necessary for its total 
score to be less than 30 for a three- response choice item, and less than 
48 for a four-response choice item. 

Twenty individuals responded to Form II of the questionnaire. The 
same procedure as described for Form I was used to determine the level 
of rejection for Form II. For an item to be rejected it was necessary for 
its total score to be less than 21 for a three-response choice item and 
less than 30 for a four- response choice item. 

An Ex ample of the Use of a Vocational Education Information Inventory 

Summary of the Study : A sixty- item instrument was designed with the 

original purpose of measuring change which might occur as a result of the 
Evaluation Systems Project having been in operation in a local school 
system. Due to delays in the development of the instrument this measure- 
ment of change was not possible. However, the instrument was provided to 
school systems participating in the project, with the suggestion that it 
might be used to measure the agreement or lack of agreement concerning the 
understandings about vocational education between certain groups within 
their school systems. The data presented in this report represent the 
results of the trial in one school in the use of the Vocational Educa- 
tion Information Inventory. 

Returns were received from 104 teachers of vocational education and 
18 administrators in the system. A t-test was computed for each item to 
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determine significant differences between administrators and teachers. 
Items were considered significant at the .05 level of significance. The 
administrators and teachers agreed with the panel of judges on all items, 
although the extent of agreement varied from item to item. 

Fifteen of the items were found to show a significant difference in 
the understanding or attitude of teachers as compared to administrators. 
The administrators had a lower average score on all 15 items than the 
teachers. The items showing a significant difference at the .05 level 
are listed below. 

4. Vocational education programs tend to result in higher national 
per capita incomes . 

5. Vocational education increases the employment rate of workers. 

*15. Cooperative work experience programs contribute little to the 

effectiveness of vocational education programs. 

17. Vocational education should emphasize the development of abilities 
that can be used for many years. 

*18. The schools need not accept the responsibility of teaching stu- 
dents how to write a letter of application for a job. 

19. Vocational education should prepare its clientele for useful 
employment. 

24. Manpower needs can be partially met through vocational education. 

*30. Employer and employee organizations should not expect tc be 

invited to cooperate with the school in planning its vocational 
education programs. 

31. Mechanization increases the need for specialized training. 

33. Manpower needs of the community should be considered in deter- 
mining the type of vocational education programs to offer. 

*36. Simulated experiences are as successful as on-the-job occupational 
experiences in training for employment. 

38. A guidance program should include effective follow-up services. 

49. Many women need vocational education experiences to vitalize 
their old skills and to learn new skills. 



* The panel of judges disagreed with these items. 
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*56* If a person is properly trained he can do well in any occupation. 

59. People should be satisfied with, and be making progress in their 
jobs as a result of their vocational education experiences. 

Ba ckground: As the Evaluation Systems Project progressed through its 

various stages it became evident that some attempt should be made to mea- 
sure the change which might occur among teachers and administrators at 
the local school level as a result of the project having been in operation 
within the school system. One of the evaluative techniques developed for 
this purpose was a "Vocational Education Information Inventory." The 
Inventory was developed after the project was well under way in local 
schools so no measurement of the individuals could take place prior to 
the initiation of the project. The data presented in this report are 
from one school system that used the Inventory close to the end of the 
project year. This report is made in an effort to show how the data 
secured from the Inventory might be used by a local school system and/or 
researchers wishing to measure change on the part of certain project 
participants and those directly or indirectly affected by a project. 

Problem. The prospect of additional financial assistance, and the 
status associated with a state, or nationally recognized research project 
may have considerable appeal to a school system. This may be due also to 
a realization of the need for change or redirection of programs. The 
question that is extremely difficult to answer is the extent of change 
which results from participation in a developmental project by a local 
school system. 

One of the more important changes which might be brought about by a 
project such as this, is a change in the attitudes and/or understandings 
of those individuals teaching and administering the local program of 
vocational education. This study was an attempt to measure if such a 
change had taken place. 

Procedures 

Devel opment o_f Instrument ; Items for the Vocational Education 



The items 



Inventory were developed by a project research assistant . ^ 
were derived from a review of literature and the experiences of the 
project personnel. Following several reviews of the instrument by the 
project director and research assistants the Inventory xvas submitted to 
a panel of vocational teacher educators and consultants. A panel of 19 

judges gave suggestions for the improvement and addition or deletion of 
certain items. 

The members of the panel of judges also marked their choices for 
each item according to the following scale: 

SA Strongly Agree or Accept 
A Agree or Accept 
U Undecided 
D Disagree 
SD Strongly Disagree 

One i i the schools participating in the project used the instrument in 
conjunction with a vocational education evaluation class which was part 
of its local evaluation effort. The results of this trial run of the 
instrument were most helpful in identifying questionable items. The 

revised instrument was then submitted to the U.S. Office of Education for 
approval . 

The instrument itself was arranged so that it could be answered on a 
mechanically scored answer sheet. From these sheets computer cards could 
be automatically produced for data analyzation and ease of storage. 

Population: When the idea of an instrument to measure change in 

the understandings and information of project participants was first 
conceived it was hoped that all of the local schools connected with the 
Project would make use of the instrument. Delays in getting the instru- 
ment to these schools resulted in rather poor usage, but the experience 
of one school and an analyzation of the data herein does provide an 
example which can be utilized in future studies. Project personnel had 
hoped the population would include all vocational education personnel, 

administrators, and, in some cases perhaps, all teachers in a local school 
system. 



Floyd McKinney, with advice from Harold M. Byram, Project leader. 
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Design ; These steps wtsre followed in conducting the study: 

j) A review of literature was made to secure appropriate 
items for the instrument. 

b) The instrument was critiqued by the project leader and 
research assistants. 

c) A trial run of the instrument was made at one of the 
participating schools. 

d) A panel of judges was selected to review and rate the items. 

e) Approval of the instrument was secured from the U.S. Office 
of Education. 

f) Local schools were given the option of using the instru- 
ment. Most of the project schools requesced copies. 

g) Answer sheets were obtained from one school so that the 
data could be presented and analyzed. 

Analysis : For those items on the instrument with which the panel 

of judges agreed, the following scores were assigned: SA=5, A=4, 

U=3, D=2, SD=1. For those items with which the majority of the judges 
disagreed, scale scores were reversed to provide comparable responses 
for computation purposes. An average score was then secured for each 
item as marked by the teachers and administrators. 

Significance of differences between teachers and administrators 
was determined by use of the t-test. Items were considered signifi- 
cant at the .05 level of significance. 

12 

Instrument for Evaluation of Citizens' Committee Activities 

As the project progressed it became evident that a great deal of 
local school staff time and effort was being expended in organizing and 
using citizens ' committees. A "Checklist for Evaluating school Citizens 
Committees for Vocational Education Evaluation" was developed in an effort 
to help local project leaders qpc? research assistants in evaluating cit- 
izens' committees in their schools from a review of the literature and the 

19 

Developed by Floyd McKinney research assistant. 
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experiences of the I13U project staff. Before final publication the Check- 
list was reviewed by one of tne local school project leaders who had been 

active in organizing several successful citizens' committees. See Appendix 
D for a copy of the Checklist. 



SPECIFIC CHANGES RESULTING FROM, OR 
RELATED TO THE LOCAL PROGRAM EVALUATION 



From the final project reports, and from year-end, depth interviews, 
a wide variety of changes were identified. Some of these could be classed 
as innovations, others as specific implementation acts or adoptions, and 
still others as specific aspects of plans. It is probable that there are 
other changes in the planning stage and therefore not reported at the 
close of the local project year. No distinction has been made between 
those which were a direct result of the evaluation project and those 
resulting in part from the project, but also associated with other factors. 
These are reported by size groupings of the schools. 

Small Schools 



School A reported the following: 

--Curricula revisions in agriculture to place greater emphasis on 
off-farm agricultural occupations and in home economics, with 
emphasis on family living 

— Enrollment increase in industrial education courses 

— Acceptance for implementation of the recommendation calling for 
released time for vocational department heads for curriculum study 

--Vocational certification of teachers of office education 

School B reported the following: 

--The development of a three-phase "Careers and Occupational Guidance 
Program.” Phases include: (1) 9th-grade level career unit in 

English; (2) teachers in specialized areas submitting list of 
materials and films to show career opportunities; and (3) 11th- 
grade unit on preparing for employment 

--The addition of a %-time counselor at both junior-high school and 
senior-high school levels 

--The selection and purchase of $5,000 worth of audio-visual materials 
for vocational guidance of students 





--Addition of a social studies course in the curriculum, with emphasis 
on occupational orientation 

— Continuance of employer interviews and advisory committees 

School C reported the following: 

— The integration of family-living units into the home economics 
curriculum 

— Standards developed for statement of behavioral goals in retailing 
from survey of employers 

School D reported the following: 

— Establishment of the position of local director of vocational educa- 
tion, and appointment of member of leadership team to the position 

--New equipment provided by industries having representatives on the 
advisory committees 

Medium Schools 



School A reported the following: 

— Advisory committees established for all vocational areas 
— Position of department head in business education created, and 
member or project leadership team appointed to the position 
--Established need for part-time assistant director of vocational 
education for the high-school level (director presently identified 
largely with post -high school program) 

— Added course in home economics for boys 
--Third teacher of home economics employed 

— Published course description booklet, for student, teachers, and 
counselor use 

School B reported the following: 

--Made six major curricular changes: 

1. Reorganized and re-structured curriculum in agriculture 

2. Combined Homemaking II and III into one course 

3. Added course, ’’Food Service Training" 

4. Reorganized Graphics I course 

5. Revised occupational arts units in grades 7-8-9 

6. Introduced simulated distributive education laboratory for 
"Sales and Retailing" course 
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--Re-planned placement and follow-up program for all students 
School C reported the following: 

A bond issue for construction of a new high school, with greatly 
expanded facilities for vocational education, was approved by the 
voters after two previous defeats. The opinion of those involved 
in the evaluation project is that changes in attitudes toward, and 
understandings about vocational education on the part of teachers 

and citizens were a major factor in bringing about this accomplish- 
ment. 

Large Schools 

School A reported the following: 

—Now have catalogue descriptions, and statement of objectives and 
course outlines of all vocational courses 

—Now have inventory of facilities and equipment for each vocational 
department 

—Expansion of curriculum through addition of the following courses: 

Food Service Training: one for boys and one for girls 

Distributive Education III 7 

Typing II 

Vocational Secretarial Training 

Business Machines 

General Office 

Home Mechanics 

Woodwork V 

Blue Print Reading 

Forestry 

Fruit Production 

—Established position of vocational coordinator and appointed a 
staff member to fill the position 

—Dropped courses in personal clothing, bookkeeping III, and secre- 
tarial practice 

School B reported the following: 

--Made survey of 1965-66 day-trade graduates (75 students) 

Made survey of drop-outs, and re-established communications with them 
--Complete compilation made of all vocational courses 



—Reports of research studies completed within the district pub- 
lished and distributed 

--Bibliography of vocational education (18 categories) prepared and 
distributed 

Project being carried out to determine necessity and feasi- 
bility of offering a course in occupations at the junior-high school 
level 



—Cooperative discussions between vocational education teachers and 
guidance counselors held to establish responsibilities and role in 
vocational guidance 

--New program for junior-high school "Early bird" and "Late bird", 
initiated 



Additional programs or courses installed are: 



Auto Mechanics 
Appliance Repair 
Cosmetology 
Vocational Printing 
Technical Drafting 
Residential Development 
Drafting and Design 
Auto Body Bump & Paint 
Hospital Occupations 



Engine Mechanics 
Machine Operator 

Hospitality Service Industries 1 year 
Quantity Cookery 
Food Merchandising 
Machinist Toolmaker 
Service Station 

Electronics Prog. & Course I & II 



School C reported the following: 

--Ground-work laid for establishing a placement service for the school 

Ground-work laid for improved orientation toward the occupational 
world 

--A follow-up of business education graduates made which has paved 
the way for an over-all follow-up study for the school 
--Conducted study of employment of high school youth in the district 
—Provided longer blocks of time for industrial arts classes 
Developed a unified arts and occupational education program 
—Auto mechanics workshop for elementary teachers 
-Procurement of industrial arts equipment for elementary grades 
through NDEA assistance 

-Workshop for elementary teachers on handicraft projects 
-Created position of elementary consultant on industrial arts 
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— Occupations advisory committees organized for office, distributive, 
trade and industrial and health 

— Independent studies conducted the following year by four teachers 
— Vocational guidance materials added 

— An interdisciplinary course established, including English, Math- 
ematics, Science, and Industrial Arts 
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III. CONCLUSIONS, RECOUMENOAIIONS and IMPUCAIIONS 



CONCLUSIONS REGARDING ATTAINMENT OF OBJECTIVES 

The conclusions that follow are organized according to the objec- 
tives that were drawn up for the project. These conclusions are supported 
in part by the records and reports of the 10 schools involved, and in part 
by the observations made and information obtained by the state project 
leader through his contacts with them. They may reflect the project lead- 
er s interest in the project and the values held by him. 

1. To further try out and demonstrate a system for the evaluation 
of vocational education on the local level which originated in 
the Michigan Project on Evaluation of Local Programs of Voca- 
tional Education. 

The system for evaluation was tried out, not in systems of all sizes 
but in those having enrollments between 2,000 and 30,000, K-12, and in all 
regions of Michigan. It was tried out in terms of activities, since all 
schools prepared plans; all had staff committees on evaluation, and 
otherwise involved a total of 317 local educators; all studied curricula 
in terms of needs for occupational education; and all schools either con- 
ducted some type of follow-up study and/or had done so in the immediate 
past, or engaged in planning for follow-up studies. The majority organ- 
ized and used a total of 45 advisory committees involving 303 citizens. 

In other words, all activities engaged in by schools in the previous 
Michigan evaluation project were tried out in the present project. 

The actual demonstration of the system was accomplished in several 
ways. One was through communication to all the schools involved, through 
progress report meetings, and a final meeting on implementation. Another 
communication was to other school people in Michigan through presentations 
by the chief investigator on 16 different occasions. The revision of 
the manual on evaluation of local programs was based in large part on 
the experience of the 10 cooperating schools, and thus also represents a 




demonstration of the system. The conclusion is that the system was tried 
out and successfully demonstrated. In cases where an activity or proce- 
dure was not carried out with success the circumstances related to or 
responsible for the lack of success have been reported. These were a 
minority however. Eight of the ten schools 5 leadership teams and/or admin- 
istrators indicated that the evaluation effort would continue. 

2. To discover and/or devise new or improved procedures for local 
program evaluation. 

The fact that each school was in some way unique in its approach or 
method of evaluation indicates that a great deal of flexibility was pro- 
vided for local initiative as to procedures used in the evaluation effort. 
Schools varied in the procedures they used, and it was expected and con- 
sidered desirable that they would vary. 

14a ny of these new or improved procedures have been reported here and 
in the revised Manual. The more numerous ones relate to follow-up studies, 
to advisory committees, and to staff involvement. Procedures not previously 
tried out include the credit workshop in one school, and the preparation 
of a catalogue of vocational courses with amplified statements regarding 
each course. 

From the standpoint of organization and structure, two improved prac- 
tices were tried out and/or demonstrated. One was the obtaining of admin- 
istrative commitment to the project, thus underlining the realistic nature 
of the effort, and consequently of the recommendations to be developed and 
made in each school. This was reinforced by administrative support given 
at crucial stages during the local project year. The other was the desig- 
nation and use of the leadership team in each school. If one member had 
to be absent from any state or local project activity the other member 
represented the team from the local school. Identification of the func- 
tions to be performed by each also seemed to be of value, so that the two 
individuals could complement each other. The conclusion is that not only 
were new or improved procedures discovered or developed, but also that the 
project tended to validate certain procedures indicated in the literature 
as desirable in an evaluation effort. 
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3, To establish a working environment in which learning of evalua- 
tion procedures for both local school personnel and potential 
leaders at state and national levels of vocational education can 
take place. 

This was accomplished through the functioning of the 10 leadership 
teams and their staff and citizen committees. The working environment, 
furthermore, extended to the workshop and was represented in the five 
other meetings held for these teams and the 34 visits made by the state 
project leader. The materials that were made available to these teams 
also enhanced the environment. 

The working environment for the state and local leaders was not as 
adequate as had been hoped for. The four research assistants assisted 
with three meetings of all teams and carried out various responsibilities 
of a leadership nature. However, a total of only 24 consultation requests 
were available to them for this type of experience. The other research 
activities in which these assistants engaged, however, supplemented these 
experiences, and presumably contributed to their growth toward a pro- 
fessional goal of filling a state leadership role. These included minor 
structured research studies, development of instruments, bibliographical 
search, and assistance with conferences. The conclusion is that the 
working environment was very satisfactory for the local leadership teams, 
and satisfactory for those receiving experience at the state level. 

4. To identify and describe the role of the consultant in program 
evaluation. 

One aspect of this objective was achieved through the role analysis 
study made and reported in a separate section of this report. The iden- 
tification of the role specifically with evaluation and with vocational 
education was not necessarily accomplished but can be inferred. 

The other aspect of the role was the nature of requests for consul- 
tants' services relative to evaluation procedures which were general, 
rather than relating to specific vocational fields. This would suggest 
that there is a need or demand for state consultative services not neces- 
sarily identified with any one of the vocational fields. 



5. To uncover situations that could be considered as potential 
research and development centers. 

As the project progressed through the year it was evident that cer- 
tain conditions within the local school produced desirable or undesir- 
able effects on the expected performance on the project. Generally, the 
enthusiasm and atmosphere for research and innovation generated and pro- 
vided by the administration was the key factor in providing the type of 
situation needed at the local level for research and development activi- 
ties. There were three schools identified, in addition to those already 
involved in another R & D project, that could be considered for such 
projects or similar ones. 



DEVELOPMENT OF LOCAL LEADERSHIP IN EVALUATION 



Good leadership at the local level was established as crucial for 
success of the evaluation in the local school. As would be expected, 
some schools had better project leaders and/or research associates than 
other schools. As the project progressed, however, it became apparent 
that the local leadership teams were growing in their ability to deal 
more effectively with staff and citizens in the evaluation effort. Evi- 
dences of this include the promotion to greater leadership responsibi- 
lities of four persons either during or shortly following the project 
year. These were in two large schools, one medium school and one small 
school. Four leaders from three other schools were asked to make pre- 
sentations regarding their local project activities on six different 
occasions in five different states. Two leadership team members have 
resumed or started new graduate programs. It can be concluded that 
leaders in most of the cooperating schools made substantial professional 
growth, particularly in competencies required in the evaluation activi- 
ties undertaken. 
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STRENGTHS AND WEAKNESSES OF THE SYSTEM 



Strengths 

The emphasis placed on goals and outcomes, rather than solely on ways 

and means resulted in perhaps the greatest strength of the project. This 

encouraged schools to look at their objectives and to attempt to determine 

the success of the program through consideration of the end product - the 

former student. Ways and means were not neglected, but the emphasis was 
on output. 

The involvement of local staff and citizens was a fundamental con- 
cept of the evaluation system and proved to be one of the more important. 
Staff members were brought together as a group, in some cases, for the 
first time to work on a common effort. The idea that those responsible 
for the program should be involved in its evaluation proved to be sound. 

Local citizens, where they were used, were enthusiastic and very helpful 
in the evaluation effort. 

Local school personnel were very receptive to the consultative ser- 
vices offered by the project. Schools frequently requested consultants 
either from the project or the Michigan Department of Education. 

A desirable strength of the project was its built-in feature of 
bringing together representatives of many different groups - staff, cit- 
izens, consultants, and administrators — to generate and sustain an 
enthusiasm for evaluation and better understanding of vocational educa- 
tion, The momentum generated appears to have been carried forward in 
most of the schools, following official termination of the project. 



Weaknesses 



The local schools were not required to develor or : vise objectives 
for vocational education programs or courses. The, >.e encouraged to do 
so, but it was not mandatory. It had been hoped that if objectives had 
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been stated quite specifically and behavioral goals stated, the attain- 
ment oi these would have been specifically assessed. Some schools paid 
a minimum of attention to objectives and this may have resulted in some 
less profitable activity on the part of these schools. This may be the 
most serious weakness of this project. 



No objective measurement was undertaken to determine the 



extent or 



change in the local school staff or citizens as a result of participation 
in the project. Subjective measures were used, and these indicated posi- 
tive results. 



Some other limitations in scope could be considered weaknesses. In 
three of the school districts there is a community college offering tech- 
nical education programs. These were not included. Another limitation 
was that very little attention was given to the evaluation of adult voca- 
tional education programs. This was due in part to the greater local con- 
cern with the secondary education programs, and in part to the jurisdic- 
tional lines existing, A strong, comprehensive evaluation would have 
included programs at both of these other levels. 



MANUAL FOR TEACHERS, ADMINISTRATORS, AND CITIZENS ON PROGRAM EVALUATION 

Evaluation of Local Vocational Education Programs, A Manual for Admin - 
istrators, Teachers, and Citizens , by Harold M. Byram and Floyd McKinney, 
second edition, was published as a result of this project and the preceding 
project, previously referred to. The Manual incorporates many of the ideas 
tried out in both proiects, as well as a few suggested in the literature, 
but which need further testing. 



NEED FOR DETERMINING GENERALI ZABILITY OF THE SYSTEM 



The original project, and this project involved only schools located 
in Michigan, The question may be raised regarding the genralizability of 
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the system to other states. Because of the crucial importance of local 
program evaluation it is recommended that the system developed and tried 
out in this project be tried out in Its same form or be modified to suit 
needs in other states. It needs to be determined whether the system is 
workable in states with different kinds of district organization, different 
structure for state leadership and/or different leadership capabilities. 
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APPENDIX a 



research and development program 

IN VOCATIONAL- TECHNICAL EDUCATION 
Michigan State University, 310 Erickson Hall 



maiUKANDUM OF AGREEMENT 

mchigarst«rl^Irs 1 itr^ P thf am ^ V ° CaCi0nal - Tech nical Education at 
District, City of 57 -^ — School 

“• — — srsr; is » . 

by a grant from the^nited^^ pertains is mainly supporte 

University under contract //0E-5-85-lll^ e ° f Educatlon to Michigan State 

Both Michigan State University and the 

District agree to carry out the researrfT"a^T"a^ — » School 

i qcc 1 , . resea rcn and development effort betHnnino- 

d ep ending on i^t ed Sta^VomTof Edu^lt • thr ° U8h 

The personnel at Michigan State University and °° ntlnuatl0n of ^“d supper I 

■frav at any tina. In 3Uc h evant th^schtmWlsMict* elUt aicher “I “ith- 
pro-rata share of funds paid. district agrees to repay the 



tte specifications of this research and 
the attachment. 



development 



endeavor are shown on 




2 . 



All research data and reports are confidential and *u 

Of the United States Office of dential and the property 

by the IISU Project Director in e 1 . n untl * formally released 
U.S.O.E. contract* conformity with the terms of the 



development 'progra^are thIS reSeard > - 

viding consultant services. Subject toTTS E aT® 68 f Pr °‘ 

“ n » — ">■ «<- SnXt XrinKr^. 




/ s/ 



/s/ 

Peter G. Haines, Director 
Research and Development Program 
in Vocational -Technical Education 
Michigan State University 



Superintendent or Authorized 
Representative 

School 

City State 



Financial Considerations : 



MSU will provide funds to the local distract to assist the school to 
bear the added costs of the project. Payment will be made in the form 
of an honorarium of $ ___ ______ __ in recognition of the appointment of a 

research associate. Each MSU contribution will be paid in installments 
on November 30, 1966, March 31, 1967 and June 30, 1967. 

Local School Personnel Responsible : 

Local director of the project: 



Address _____ Phone 

Research associate on the project: 

Address Phone 



Michigan State University Project Leader: 
Harold M, Byram 

339 Erickson Hall, Phone: 355-1837 

Michigan State University 
East Lansing, Michigan 



Project on Evaluation Sys t ems for Programs 
^Vocational Education in the PubT-fr Schools 



Description of Project: 

evaluation; ^^obdiscove^or^eTOlop^dditional^o* 16 ? 2* ^ ^ 

evaluation; to T^uTa' ZT* ° f 00n8ulta “^ 

procedures; and to uncover situations'that" 6 *iji 5 rainin8 in evaluation 
tial BSD centers. situations that could be considered as poten- 




— Provide conferences and workshops to 
competencies of representatives from 
travel and subsistence costs. 



further develop evaluation 
cooperating schools, including 



'^^LaTa^r^n^'^^f 0 ” ° f education 

ating schools. national fields, including visits to cooper- 



-Provide short-time consultant help as needed and/or requested. 
-Assist in locating part-time assistants, as requested. 

'« lnstruraents and related 

reC ° rd3> rep ° rtS and other <>« a from coopera- 

--Prepare and publish report on the project, 

R esponsibilities of t he Cooperating School * 

3S “ Ho* with s.„„l 

commensurate witl/thebsize^ofba^ 16 ? 60 ^*^ re3earch associate 
tional education program! ° f SCh ° 01 “* SiZe and sc0 » a of voca- 

-Provide essential clerical s-.rvices and facilities. 

--Keep appropriate records of activfMoo 
local project. ctivitxes and prepare report on the 



’raeds? e<IUeStS f ° r and utilize consultant 



service appropriate to 
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APPENDIX B 



EVALUATION SYSTEMS PROJECT 
Questions for Interviewing Local Administrator 



1, What do you see as the future or continuation of your effort in local 
program evaluation? 

2, To what extent has the evaluation project in your school accomplished 
what you hoped it would when it was decided that your school would 
cooperate? 

3, Were there any inhibiting factors that arose during the project, that 
proved difficult to deal with? If so, what? Any enhancing factors? 
If so, what? 

4, To what extent was the communication process between your project 
leader and/or research associate and the teaching and administrative 
staffs regular and adequately informative? 

5, How has the school board been involved in the project? 



EVALUATION SYSTEMS PROJECT 
QUESTIONS FOR INTERVIEW 
WITH 

LOCAL PROJECT LEADER AND RESEARCH ASSOCIATE 



When you made your plan for your evaluation project you set up 
specific objectives to be accomplished. How well has your Evalua- 
tion SysVms project helped you to reach these objectives? 

Have you, since beginning the work of the project, changed to objec- 
tives of your local project or think now that they should be changed? 
If so, how? 

Do you feel that the staff involvement in your school was greater 
than required, sufficient, or insufficient? 

The carrying out of the project through committees necessitates time 
for planning, meeting, and working. What are some of the inhibiting 
factors in providing staff with planning, meeting, and working time? 
What are some of the enhancing factors? 

Do you feel that the philosophy of vocational education held by your 
staff has changed during the year? If so, in what ways? 

Is there a tangible improvement of community attitudes toward, and 
understanding of the vocational and technical education needs of 
the school and community? If so, in what ways? 

To what extent has the project stimulated or developed community 
understanding of vocational-technical education? 

What do you see as the future or continuation of the effort at local 
program evaluation? 

What changes from the procedures that you followed would you recommend 
for another school that might like to conduct an evaluation of its 
program? 

What is your evaluation of the consultative service received from 
Michigan State University staff members on the project? 

Would you have liked more instrumentation? 



appendix c 



CHECKLIST FOR EVALUATING SCHOOL CITIZENS ' COMMITTEES 
FOR VOCATIONAL EDUCATION EVALUATION 



Directions: Mark each question in relation to the oper- 

ation of the citizens’ committee evaluating 
vocational education in your school system. 



YES 


NO 
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1. Is provision made for the permanence and continuity 
of the committee? 

2. Has the local board of education developed policies 
relative to: 

a. citizen committee functions 

b. number of committee members 

c. method in which members are chosen 

d. method of replacing members when their terms expire 
or when vacancies occur 

e. procedures for communicating with school personnel 
and board members 

f. relationships of the committee to board members, 
administrators, teachers, students, and the com- 
munity 

3. Are members of the citizens* committee nominated by the 
school staff? 

4. Does the board of education approve the appointment of 
individuals to the citizens ' committee? 

5. Are members selected because of their: 
a. interest 

b • knowledge 

c. ability 

d. willingness to serve 

e. "segment or interest group" representation 

6. Members are notified of their appointment by: 

a. letter 

b. personal visit by a school representative 
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NC 
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7. Are committee members appointed for a definite length 
of time? 

8. Are committee membership replacements staggered? 

9. Does structure provide that members will not succeed 
themselves? 

10. Is the chairman elected from the ranks of the committee? 

11. Is the school represented by an ex-officio member of 
the committee? 

12. Are committee members aware of the specific purposes 
for which the committee is formed? 

13. Are committee members informed well in advance of meet- 
ing dates and places? 

14. Are committee meetings held only when sufficient need 
exists to merit the meeting of the committee? 

15. Is publicity provided for committee activities? 

16. Are formal committee recommendations written? 

17. Are committee recommendations channeled through appro- 
priate school authorities? 

18. Does the citizens' committee accept the philosophy that 
its duty is that of recommendation, not of legislation 
or administration? 

19. Does committee membership include representatives of all 
social classes in the community? 

20. Are young adults represented in the committee number ship? 

21. Is there a feeling of need for the citizens' conmitte 
by: 

a. board members 

b. administrators 

c. school staff personnel 

22. Are research expenses of the committee paid for by the 
school? 
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YES 


NO 


















* 

i 

¥ 

























23. As related specifically to vocational education evaluation, 

does the citizens' committee: 

a. make suggestions concerning the objectives of the 
school program of vocational education 

b. advise on curriculum planning 

c. make suggestions regarding the extent to which 
occupational skills should be taught in the school 

d. help develop instruments for follow-up of students 

e. help conduct surveys of former students, parents, 
and/or employers 

f* help acquaint the community with the needs of voca- 
tional education 

g. provide feedback to employers and other citizens 

h. help assemble and analyze materials relative to 
program evaluation 

i. help develop procedures for follow-up of students 

j. help interview former students, parents, and/or 
employers 



S 


P 


N 







































Directions: Mark the remaining questions using the following 

key: 

S - satisfactory progress in this area 
P - only partial success in this area 
N - not satisfied with progress in this area 



24. Are committee members fully aware of the purpose of 
the citizens ' committee? 

25. Do committee members possess a working knowledge of the 
philosophy and objectives of vocational education? 

26. Is an information program provided that will keep com- 
mittee members abreast of new vocational education devel- 
opments? 

27. Is the committee formally organized? 

28* Does the school representative perceive his duty largely 
as that of requesting rather than giving advice? 

29. Are committee members well oriented in the principles 
of vocational education? 



- 88 - 



30. Do school personnel work with the citizens * committee 
when the citizens' committee is concerned with a specific 
portion of the school curriculum for which a school staff 
member is responsible? 

31. Are committee members Invited to school functions? 

32. Are committee members considered a part of t^e "school 
family?" 

33. Are preconceived notions sublimated or minimized in 
actions of the citizens 1 committee? 

34. Does the committee study and debate the issues before 
making final recommendations? 

35. Can the committee request school personnel to conduct 
research? 

36. Does the school provide adequate research services for 
the citizens 5 committee? 





APPENDIX D 



CAPSULE DESCRIPTIONS OF LOCAL PROJECTS* 



Alpena 



A community of 14,682, Alpena is located in Northeast Michigan on 
the shore of Lake Huron. The school district covers considerable rural 
area and includes substantial industrial development. The R-12 enrollment 
was 8,340. Business education, home economics, agricultural education, 
industrial education, and cooperative education are offered in the school's 
secondary vocational education program. There is a technical education 
program in the local community college, but not included in the evaluation 
project. 



the vocational department heads. Citizen advisory committees were not 
used, but plans were laid for establishment of these committees, which 
have since been carried out. 

Evaluation activities included participation in ’ three-school meeting 
regarding evaluation of vocational education, a folic /-up study of '65- 
'66 graduates, establishment of a card file on all graduates, and a review 
of departmental objectives. 

Benton Harbor 

Benton Harbor is located in Southwest Michigan on the shore of Lake 
Michigan. The city had a population of 19,136, with the district serving 
an estimated population of 47,492. Urban, suburban, and rural segments 
- of population are represented within the district. The area is served 
with a relatively large amount of heavy and light industry, and a heavy 

* 

Does not include Stephenson, which was not officially a part of the pro- 
ject, although consultant resources were made available to, and meetings 
were attended by its project leader. 



Staff members most actively involved in the evaluation 'Effort were 
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concentration of fruit production. It has an important summer tourist 
and vacation business. The K-12 enrollment was 11,900. The school's 
vocational education program includes business and distributive education, 
home economics, agricultural education, industrial education, and coopera- 
tive education. Eleven staff members and administrators constituted the 
evaluation project central committee. Several meetings of this committee 
were held, with the project director meeting regularly with individual 
representatives of each of the vocational areas. 

Ten different citizens advisory committees were used, involving 63 
different community citizens. 

The evaluation activities included the development of basic objec- 
tives for each vocational class, course outlines in each vocational sub- 
ject, and analysis of vocational facilities and equipment, a follow-up 
survey of one class of former students, a look at the qualifications of 
the vocational teachers, and the use of citizens' advisory committees. 

Big Ranids 



Located on the Muskegon River at the intersection of highways M~20 
and US-131, Big Rapids is a community with a population of about 12,000. 
The area is served by several industries and businesses and is the home 
of Ferris State College. Total enrollment of the Big Rapids school sys- 
tem was about 2,500, with almost 800 of these in high school. The voca- 
tional education program includes offerings in home economics, agricul- 
tural education, non- reimbursed business education, and cooperative edu- 
cation. 

Rfiprescntativcs from the school's administration and faculty members 
from the departments of home economics, business education, industrial 
arts, and guidance and counseling served as staff committee for a total 
involvement of 20 members. Many of the staff meetings were informal, 
with no released time being given for such meetings. 

Big Rapids did not use citizens in any capacity in its evaluation 
effort, although there is a past history of some citizen involvement. 
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Consultant services were utilized in developing the questionnaire 
for the follow-up of former students and in preparing the data for com- 
puter analyzation. The chief evaluation effort was this follow-up study 
of the classes of 1966, 1965, and 1962. 

Corunns 

Corunna, a town of 2,746 persons, is centered between Flint, Lansing, 
and Saginaw. Some small industry exists in the area, but for the most 
part agriculture is the dominant economic force. The larger percent of 
the K-12 student body of 2,460 was from the rural area. The school's 
vocational education program includes offerings in office and distributive 
education, home economics, agricultural education, and industrial education. 

Seventeen of the school's teachers and administrators served on a 
staff committee and met five times. Four of these meetings were held 
jointly with .:he citizens' advisory committee. Twenty-four community 
citizens served as members of the vocational advisory committee. All 
staff members and citizens usually met jointly and then continued their 
work in the areas of trade and industry, distributive, office, agriculture, 
and health and food service. Both staff ard citizen committee members 
previewed and assisted in the analysis of the results of a questionnaire 
which was a part of a follow-up study of former students. The final 
meeting of the joint citizen-staff committee was a banquet held at one of 
the school buildings. The school administration and the local board of 
education were in attendance at the meeting to hear the recommendations 

r 

of the various citizen- staff committees. 

School administrators were involved in the evaluation effort as mem- 
bers of the citizen-staff committee. The local project leader was assis- 
tant superintendent of schools and the research associate was the director 
of vocational education. 

Consultant services were utilized in the development, scoring, and 
analyzing of the follow-up study, in the specialized areas of vocational 
education, and in the motivation and stimulation of the citizen-staff 
committee. 
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The staff committee reviewed the objectives of the vocational educa- 
tion department at its first meeting. Much of the evaluation effort 
centered around a follow-up of the 1961 and 1965 high school graduates. 

A questionnaire type of follow-up was used. 

Released time was given vocational staff members to visit three 
places of employment in their relate a ...» 

was used for discussion purposes when interviewing employers, 

Fremont 

A town of 3,384 population, Fremont is located in Western Michigan 
about 30 miles from the shore of Lake Michigan, Fremont is basically a 
one-fndustry (food processing) town, with the school area comprising 
considerable rural area. A substantial number of people commute to sur- 
rounding areas for work. The K-12 enrollment was 2,600. The school's 
vocational education program includes offerings in home economics, agri- 
cultural education, industrial education, non- reimbursed business educa- 
tion, and cooperative education. 

All vocational staff members were informed of the project and its 
purpose. The most involvement was limited to two administrators, the 
project leader and research associate, and the vocational department heads. 

Twenty- four community citizens were involved as members of four dif- 
ferent vocational citizens advisory committees. One department met indi- 
vidually with members of the advisory committee. The coranittees for the 
other three departments met for an average of at least three times. 

The evaluation activities included a survey of local secretaries 
concerning the value of their high school business education courses, a 
survey of office and retail employers, a survey of a 9-11-grade students 
to determine t&sir interest in home economics, a survey of past vocational 
agriculture graduates, a survey of current vocational agriculture students, 
and a start toward a survey concerning the interest in shared-time voca- 
tional classes among secondary schools in the county. 
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Hillsdale 



Hillsdale, a community of 7,962, is located in Southern Michigan, 
about 13 miles from the Michigan-Ohio state line. The area could be 
classified as rural -indust rial. The K-12 enrollment was 2,765. The 
school ! s vocational program consists of home economics, agricultural 
education, non-reimbursed business education, industrial education, and 
cooperative education. 

All vocational personnel were involved in the evaluation effort either 
as members of the central evaluation committee and/or as members of the 
sub-committees . 

Citizens were used on three advisory committees. Their efforts were 
directed toward providing advice on course content and on the suitability 
of facilities and equipment. Consultant services of Michigan State Univer- 
sity were used. 

Evaluation activities included a follow-up study of graduates, a 
questionnaire study concerning hiring practices of metal processing employers, 
and evaluation of the vocational facilities. 

Lansing 

Located in South Central Michigan, Lansing is a city with a population 
of 120,500, The city serves a much larger metropolitan area, is the site 
of the state capitol, is adjacent to East Lansing — the home of Michigan 
State University — and has a relatively large amount of industry, including 
the home plant of General Motors Oldsmobile Division. The Lansing school 
system enrolled 32,109 in grades K-12 and offered vocational education 
programs in business and distributive education, home economics, industrial 
education, cosmetology, food services, and cooperative education. The 
local community college provides technical education programs. These were 
not included in the evaluation project. 

Faculty involvement totaled 90 persons. However, only a small pro- 
portion participated in a totally active capacity. The board of educa- 



* 
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tion and school administrators were informed and involved where possible. 

Thirteen different vocational citizens' advisory committees, involving 
109 community citizens participated in the evaluation effort. These com- 
mittees were already a part of the operation of the school's program and 
normally meet once a year. 

The evaluation activities included a survey of the 1965-66 day-trade 
students, a dropout survey, high school curriculum choice study, dissem- 
ination of research, bibliographies of vocational education studies, infor- 
mation about sources of occupational information, role of vocational guid- 
ance, course descriptions and curriculum and instructional techniques 
information sheet. 

Niles 



Niles had a population of 13,842, and is located in the Southwestern 
part of Michigan, about 6 miles from the Michigan- Indiana line. A large 
number of the population find employment in the industries in the imme- 
diate Niles area and in the large industrial complex of South Bend. The 
Niles school system served 6,510 students in the K-12 grades. Business 
education, home economics, agricultural education, industrial education, 
and cooperative education are offered in the vocational education program. 

Twenty-seven of the school's administrators and staff served on the 
staff evaluation committee. Seven different meetings were held with some 
released time provided the staff. School administrators were actively 
involved in the project through their participation on the steering com- 
mittee. Consultant services utilized by the school were mainly those con- 
nected with the Michigan Department of Education. 

Ten different citizens' advisory committees involving 68 different 
citizens were formed to assist in the evaluation effort. These committees 
held a total of 50 meetings and assisted in the evaluation of objectives, 
curricula, and facilities. 

The evaluation activities centered in these advisory committees, but 
also included departmental descriptions of programs, departmental self- 
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evaluations, a student evaluation of guidance and counseling services, 
development of philosophy and objectives for occupational education, a 
study of the feasibility of a shared-time program, a study of organiza- 
tional patterns, a study of students enrolled, not enrolled, and partici- 
pation in a three-school meeting regarding evaluation of vocational educa- 
tion* 

Sault Ste. Marie 



Sault Ste. Marie is the one school located in the Northern Peninsula. 

It had a population of 18,722 and a principal industry is the ship janal 
which carries the largest annual tonnage of any in the country. It is 
located across the river from Sault Ste. Marie* Ontario, Canada where 
many Michigan citizens are employed. 

The school had an enrollment of 4,700 K-12 grade and offers programs 
in agricultural education, business education, home economics and indus- 
trial education. A staff evaluation committee of 15 members was used in 
planning and conducting activities, in addition to a general citizens' 
committee meeting jointly on one or more occasions. Staff discussions 
were held, which led to a reportedly greater degree of agreement in phil- 
osophy and objectives of vocational education. Part of the staff committee 
participated in a three-school evaluation conference. 

An instrument was developed which was aimed at measurement of student, 
pupil, and staff attitude toward the world of work, and toward vocational 
education programs at the secondary level. The instrument was administered 
late in the school year. This study had not yet been summarized at the 
time of preparation of this final report on the project. 

Waterford Township 

Waterford Township Schools is one of two school systems serving the 
area west of Pontiac, a city of 82,223. Pontiac is Northeast of the 
•Detroit Metropolitan area. The Waterford Township School system served 
a K-12 enrollment of 16,755. The vocational education offering includes 
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business and distributive education, home economics, health occupations 
education, industrial education, and cooperative education. 

Several meetings of school administrators and vocational teachers 
were held, but no formal staff committee was organized. Administrators 
and staff members were informed of evaluation events by means of a ,f World 
of Work Newsletter," by many informed contacts, and by various staff 
meetings. 

Citizens were not involved in the evaluation effort during the project 
year, although they had been previously involved in program planning acti- 
vities. 

Consultants from the state colleges and universities, the state depart- 
ment of education, and business and industry were utilized. 

A credit workshop on evaluation of vocational education served as the 
core of the school's efforts toward evaluation of its vocational educa- 
tion program. A summary of this workshop appears elsewhere in this report. 
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THE ROLE OF THE CONSULTANT IN 
EVALUATION OF LOCAL VOCATIONAL EDUCATION PROGRAMS 



Summary of the Study 



Two questionnaire forms were developed to ascertain the role of the 
vocational education evaluation consultant. Form I was designed for 
university vocational teacher educators and state consultants of 
vocational education. Form II was responded to by local prolect 
leaders and research associates from the schools participating in 
the Michigan Vocational Education Evaluation Systems Prolect. The 
conclusions that follow are drawn from the sampled consultants and 
consul ters and nay not necessarily be representative of all consul- 
tants and consulters. Of the 49 questionnaires distributed to the 
consultants 31 were returned, for an 83 percent return. Twenty of the 

22 consulters receiving questionnaires returned the instrument^ for 
a 91 percent return. 9 



Analysis of the data led to the following conclusions in regard to 
the five questions for which the study sought information. 

1. What is a consultant? 



There was agreement between consultants and consulters on the 
concept that a consultant on vocational education evaluation is 
primarily a resource person. While the consultant as an expert 
was held second in importance by consultants themselves, this was 
somewhat subordinated by consulters to the concept of the consul- 
tant as a process person* 



Over one-half of the consultants responding to the questionnaire 
had spent less than 20 percent of their time working as consul- 
tants. About 20 percent, however, hsd spent 80 to 100 percent of 
their time serving in a consultative role. 



Consultants had spent most of their time with administrators 
with teachers following closely i n second place. The least amount 
of time had been spent with other consultants and advisory conmit- 
tees but with local groups receiving somewhat more time. 

By far the greatest number of the responding consultants have 
been spending most of their time at the local level. In order of 

fairly consistently decreasing importance are the state, regional 
and national levels. 6j 
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The planning stage is when most consultants 1 services have been 
used Of less importance are the evaluating* implementing, and 
innovating stages. 

The consultants have spent most of their consulting time in the 
capacity of furnishing material and resources- The least amount of 
time has been spent as an expert on evaluation. Of less importance 
is assisting persons in solving their professional problems. 



2. Is there a difference in the way the consultant and consulter 

perceive the role of the consultant in vocational education evalu- 
ation? 

The rank difference coefficient of correlation between consul- 
tant and consulter was relatively high for all areas of the study 
questionnaire* Highest agreement, *904, was related to the work 
of the consultant in evaluation and the lowest, *692, in regard to 
the consultant's abilities and characteristics.. The degree of 
agreement on the consultant's traits and the consultant's activities 
was 825 and .832, respectively. 



3, What abilities and characteristics should the consultant possess? 

The responding consultants and consulters felt the abilities and 
characteristics of the consultant should be oriented toward planning, 
assisting, and recommending rather than making decisions or making 
and presenting judgments to and for the group- 

Regarded as most important were the abilities to plan, to convey 
clear thinking, to maintain a close working relationship with group 
leaders or directors, and to keep up to date on materials- 

The most important characteristics were to be able to maintain 
rapport within the group, to be democratic, able to accept people* 
and to grasp the scope of a situation- 



4. What activities can the consultant perform to stimulate evaluation? 

The consultants and consulters generally agreed on the consul- 
tant's activities in vocational education evaluation. Generally 
these activities involved the areas of planning, suggesting, 
recommending, listening, criticizing, stimulating, organizing, 
clarifying, and synthesizing- 

The most frequent activities reported by consultants and consul- 
ters were collecting and recommending resources for use by people 
involved and listening to problems and plans of groups of individual 
participants,. 
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5. What guidelines may be used to direct the consultant in his work? 



The consultant should 

--diligently prepare himself and furnish resource materials 
for the consul ters 

— have a broad background of experience in his field before 
serving as a consultant 

— personally visit the area and the people connected with the 
problem or situation 

— be dynamic, practical s democratic, sympathetic, and a good 
organizer 

--assist the group to arrive at a solution to their problem 
by offering a variety of the best possible solutions to 
the problem 

— assist in setting goals 

— assist in interpretation of information 

— keep the group informed 

— help the group move toward making a decision 

--evaluate group goals, processes, and decisions 



Background of the Study 

Experience of the staff associated with the Evaluation Systems Project 
for Local Programs of Vocational-Technical Education resulted in 
increased interest in attempting to determine the role of the consul- 
tant* This interest resulted in part from the experience of the project 
personnel in the performance of consultative activities for the loc.3 1 
schools associated with the project* The local schools were also using 
consultants frcm other sources, especially those with the Michigan State 
Department of Ed'acation. Most of the study was conducted by a project 
research assistant with the cooperation and assistance of the project 
director and other project research assistants. 



Problem 

At times vocational education evalution consultants may not be utilized 
effectively because they, or the people who use their services, do 
not have a clear concept • f their role . This lack of knowledge concern 
ing the role of the consultant may result in a less capable performance 
than possible if the consultant and consulter were to better understand 
the situation.. The problem, then, becomes one of determining the role 
of the consultant in vocational education evaluation. This study 
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attempted to answer the following questions: 

1* What is a consultant? 

2 • Is there a difference in the way the consultant and consulter 
perceive the role of the consultant in vocational education 
evaluation? 

3 # What abilities and characteristics should the consultant possess? 

4# What activities can the consultant perform to stimulate evaluation? 

5# What guidelines may be used to direct the consultant in his work? 

A consultant was defined as a person outside the school staff and 
its clientele that might be affected directly by decisions; who has had 
experience and/or education that qualify him to give advice , information, 
interpretations and/or suggestions regarding alternative courses of 
action or procedures, and/or who has skill in helping professional people 
to identify and work through their problems to a satisfactory solution. 

Consultant types were recognized as those identified by Savage.* 
These include the "expert,” the "resource person," and the "consulter"— 
one who utilizes the services of the consultant. 

Procedures 

Development of Checklist 

To determine how the consultant and the consulter view the role of 
the consultant on evaluation in vocational education, two questionnaires 

1 ■ 

From William W. Savage, Consultative Services to Local School Systems . 
(Chicago, Illinois: Midwest Administration Center, 1959). 
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were developed , Questionnaire Form X was developed to determine how 
consultants viewed their role and Form II was utilized to determine how 
consulters viewed the role of the consultant. Items for the question- 
naires were developed after conducting a search of the literature on the 
role of the consul tanc The questionnaires were then reviewed by the 
project director and research assistants. The questionnaires were 
revised and submitted to a jury of experts (Michigan teacher educators 
and consultants in vocational education),, Using their suggestions, 
further revision was made and the forms were resubmitted to the same 
jury. After further revision the forms were submitted to and approved 
by the U.S. Office of Education, 

The, Populations 

The population used for Form I included 31 vocational educators in 
Michigan universities and consultants in the Michigan State Department 
of Education. Seventeen of the returns came from vocational educators 
at Michigan State University, Wayne State University, Central Michigan 
University, Western Michigan University, Eastern Michigan University, 
and Northern Michigan University The consultants from the Michigan 
Department of Education were all employed in the Vocational Education 
Division. 



Design 

The study was conducted by a mailed questionnaire. The major portion 
of the questionnaire was designed so that the respondent could easily 
check one of several responses to an item or check a response indicating 
his agreement or lack of agreement, the importance of the item, or the 
frequency with which the item should be done. 

The following steps constitute the general design of the study. 

1. A review of literature to determine items for the questionnaires. 

2. Utilize the Evaluation Systems Project 5>taff in an advisory 
capacity in developing the questionnaires 

3. Have a jury of experts review the questionnaires. 

4. Utilize local project leaders and research associates connected 
with the Evaluation Systems Project and vocational educators in 
Michigan Universities and the Michigan State Department of 
Education to respond to the questionnaire. 



The plan for the study and instruments were developed by Joan Quilling, 
who also gathered the data and made a preliminary analysis- The 
analysis was completed and the report prepared by Floyd McKinney. Both 
were research assistants on the project under the supervision of 
Harold M. Byram, project leader. 



Analysis 



Thirty-one individuals responded to Form I of the questionnaire „ 
Assuming the respondents would equally divide their choices, the items 
with three- response choices would have about ten responses per choice, 
and the items with four- response choices would have about eight 
responses per choice. If a rating of 2-1-0 is assigned to the items 
with three- response choices and a rating of 3- 2-1-0 to the items with 
four-response choices the total score for such items would then be 
30 and 48 respectively. For an item to be rejected it was necessary 
for its total score to be less than 30 for a three-response choice 
item, and less than 48 for a four-response choice item. 

Twenty individuals responded to Form II of the questionnaire. The 
same procedure as described for Form I was used to determine the level 
of rejection for Form II. For an item to be rejected it was necessary 
for its total score to be less than 21 for a three- response choice 
item and less than 30 for a four-response choice item. 
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APPENDIX A 



£ 

Summary of the Results for Questionnaire Form I-- Consultants 

f II 

*' V 

[ l 

1. A consultant’s work on evaluation, as viewed by consultants 

v- 

l 









Number Selecting Each Choice 






Agree 


Partially 

Agree 


Disagree 


No 

Answer 


Question 


No. 


Score No. 


Score 


No. 


Score 


No. 


1. He should give answers to prob- 
lems which arise if he has the 
answer. 


13 


26 


14 


14 


2 


0 


2 


2. He should provide resource 
materials. 


26 


52 


4 


4 


1 


0 


0 


3. He should attempt to give no 
concrete answers to problems, 
only suggestions. 


6 


6 


15 


30 


10 


0 


0 


4. If a consultant sees a solution 
to a problem, he should vol- 
unteer the answer. 


9 


18 


19 


19 


3 


0 


0 


5. A consultant should offer a 
variety of problem solutions 
and allow a choice of answers. 


21 


42 


9 


9 


0 


0 


1 


6. A consultant should be concerned 
with a group's interactions and 
its reactions to a situation. 


25 


50 


6 


6 


0 


0 


0 


7. A consultant should help the 
group arrive at a definite 
answer to a problem. 


12 


24 


17 


17 


2 


0 


0 


8. A consultant should not give 
suggestions towards answers he 
feels are best for the group.* 


1 


1 


8 


16 


22 


0 


0 


9. A consultant should be highly 
skilled in human relations. 


27 


54 


4 


4 


0 


0 


0 


10. A consultant should have a broad 
background of experience before 
acting as a consultant. 


22 


44 


9 


9 


0 


0 


0 



f 

> 



* Not accepted 
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Consultant abilities and characteristics consultants think 
they should possess 



2 . 



Number Selecting Each Choice 







Always 


Usually 




Sometimes 


Not 






Important 


Important 




Important 


Important 


Abilities 


(Essential) 
No . Score 


(Desirable) (Acceptable) (Undesirable) 
No. Score No. Score No. Score 


1. 


To plan 


21 


63 


10 


20 


0 


0 


0 


o 


2. 


To be able to convey clear 


















thinking to others 


28 


84 


3 


6 


0 


0 


0 


o 


3. 


To give solutions to problems* 


1 


3 


16 


32 


11 


11 


3 


o 


4. 


To chair the group consulting 


















him 


13 


39 


14 


28 


4 


4 


0 


o 


5. 


To interpret information 


15 


45 


15 


30 


1 


1 


0 


o 


6* 


To make decisions for the group* 


0 


0 


1 


2 


11 


11 


19’ 


o 


7. 


To assist in setting group 
















8. 


goals 


12 


36 


14 


28 


5 


5 


0 


o 


To be quick and to the point 
on matters in order to keep the 
group moving 


8 


24 


10 


20 


12 


1 


o 








12 


9. 

10. 


To help groups and individuals 
move toward decision making 
To make and present judgments 
or evaluations to and for the 


15 


45 


13 


26 


3 


3 


0 


0 




11. 


group * 

To evaluate group or indivi- 


6 


18 


6 


12 


15 


15 


4 


0 


12. 


dual goals and purposes 


7 


21 


14 


28 


10 


10 


0 


0 


To recommend ways that the group 
can solve its problems 


12 


36 


15 


30 


4 


4 


0 


0 


13. 


To maintain a close working re- 
lationship with group leaders 




















14. 


or directors 


23 


69 


7 


14 


1 


1 


C 


o 


To keep people informed of what 
happens 


18 


54 


10 


20 


3 


3 


0 


o 


15. 


To keep up to date on materials 


25 


75 


6 


12 


0 


0 


0 


o 


Characteristics 














u 


1. 


Be dynamic and cause things to 
happen 


6 


18 


19 


38 


6 


6 


0 


o 


2. 


Be a good public relations 




















repre s enta t ive 


18 


54 


11 


22 


2 


2 


o 


o 


3. 


Be able to determine the in- 
terests and creativeness of 
personnel 


15 


45 


11 


22 


5 


5 


o 


o 


4. 


Be practical 


17 


51 


12 


24 


2 


2 


o 


o 


5. 


Be able to maintain rapport 
within the group 


23 


69 


8 


16 


0 


0 


o 


o 


6. 


Be democratic 


20 


60 


11 


22 


0 


0 


o 


w 

o 


7 * 


Be sympathetic 


12 


36 


15 


30 


4 


4 


o 


w 

o 


8. 


Have expertise relative to 














V 




subject matter 


18 


54 


13 


26 


0 


0 


o 


n 


9. 

10. 


Be able to accept people 
Be able to grasp the scope of 


22 


66 


9 


13 


0 


0 


0 


0 




a situation 


22 


66 


9 


18 


0 


0 


0 


0 



*Not accepted 




3. Frequency of activities 9 reported by consultants 



Number Selecting Each Choice 



i 

Activities 


No. 


Often 

Score 


Sometimes 
No. Score 


Practically 
Never 
No. Score 


1. Make visits to the area where 
the problem developed 


13 


26 


17 


17 


1 


0 


2. Help in the pre-planning and 
setting up of problems or plans 


15 


30 


14 


14 


2 


0 


3* Conduct or assist in evaluation 
of programs or plans 


11 


22 


18 


18 


2 


0 


4.. Ask for an evaluation of your 
own consultant service* 


4 


8 


13 


13 


14 


0 


5. Give suggestions when requested 


21 


42 


10 


10 


0 


O 


6. Help to keep projects or plans 
moving toward their objectives 


20 


40 


10 


10 


1 


0 


7. Collect and recommend resources 
for use by people involved 


24 


48 


7 


7 


0 


0 


8. Listen to problems and plans 
of group or individual par- 
ticipants 


24 


48 


7 


7 


0 


•3 


9. Criticize plans or 
procedures 


5 


10 


23 


23 


3 


0 


10. Analyze and interpret the data 
or information involved 


13 


16 


17 


17 


1 


0 


11 o Stimulate thinking on problems 
and plans 


20 


40 


11 


11 


0 


0 


12. Help organize planning groups 


8 


16 


19 


19 


4 


0 


13. Help organize plans of action 


13 


26 


17 


17 


1 


0 


14. Demonstrate an idea for clari- 
fication 


8 


16 


18 


18 


5 


0 


15. Help to decrease friction within 
groups or among individuals 


7 


14 


17 


17 


7 


0 


16. Participate in group discussions 


23 


46 


8 


8 


0 


0 


17. Help clarify group or indivi- 
dual thinking 


16 


32 


15 


15 


0 


0 


18. Sythesize material in order 
to make recommendations (pull 
things together) 


12 


24 


19 


19 


0 


0 


19. Interpret the developments 

within a meeting to all members 
involved 


12 


24 


17 


17 


2 


0 


20. Prepare for visits before seeing 
people involved 


19 


38 


10 


10 


2 


0 



* Not accepted 
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Appendix B 



Summary of the Results for Questionnaire Form II — Consulters 
1* A consultant *s work on evaluation as viewed by consulters 



A AVAA 



tcutxaix) ngicc 



Question 

1. He should give answers to 
problems which arise if he 
has the answers 

2. He should provide resource 
material 

3. He should attempt to give no 
concrete answers to problems, 
only suggestions 

4. If a consultant 3ees a solu- 
tion to a problem, he should 
volunteer the answer 

5. A consultant should attempt 
to offer a variety of problem 
solutions from which a choice 
may be made 

6* A consultant should be concerned 
with a group’s interactions and 
with an individual or group’s 
reactions to a situation 

7. A consultant should help the 
group or individual arrive at 
a definite answer to a 
problem 

8. The consultant should not give 
suggestions towards answers 

he feels are best for the 
group or individual* 

9. A consultant should be highly 
skilled in human relations 

10* A consultant should have a 

broad background of experience 
in his field before serving 
as a consultant 



No. Score No. 



14 

18 



13 



19 



11 



15 



17 



28 

36 

12 

18 



26 



38 



22 



2 

30 



5 

1 

11 

8 



1 

5 



Score 



5 

1 

11 

8 



1 

5 



^ 

uiaagtce 



No. Score 



1 

1 

3 

* 3 



18 

0 



0 

0 



0 

0 



34 



*Not accepted 
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2. Consultant abilities and characteristics desired by consulters 



Number Selecting Each Choice 







Always 

Important 


Usually 

Important 


Sometimes 

Important 


Not 

Important 


Abilities 


(Essential) 
No. Score 


(Desirable) 
No. Score 


(Acceptable) 
No. Score 


(Undesirable) 
No. Score 


1 . 

2. 


To plan 

To be able to convey clear 


15 


45 


3 


6 


2 


2 


0 


0 


3. 


thinking to others 
To analyze problems and/or 


18 


54 


2 


4 


0 


0 


0 


0 


4. 


programs 

To give solutions to 


13 


39 


7 


14 


0 


0 


0 


0 


5. 


problems 

To chair the group 


2 


6 


11 


22 


6 


6 


1 


0 




consulting hint* 


1 


3 


7 


14 


4 


4 


8 


0 


6. 

7. 


To interpret information 
To make decisions for the 


13 


39 


7 


14 


0 


0 


0 


0 


8. 


group 

To assist in setting 


0 


0 


1 


2 


7 


7 


12 


0 


9. 


group goals 

To be quick and to the point 
on matters in order to keep 


5 


15 


8 


16 


6 


6 


1 


0 


10. 


the group moving* 

To help groups and individuals 


2 


6 


4 


8 


10 


10 


0 


0 


11. 


move toward decision making 
To make and present judgments 
or evaluations to and/or 


9 


27 


6 


12 


5 


5 


0 


0 


12. 


for the group* 

To evaluate group or indivi- 


1 


3 


4 


8 


9 


9 


6 


0 


13. 


dual goals and purposes 
To recommend ways that the 


4 


12 


7 


14 


8 


8 


1 


0 


14. 


group can solve its problems 
To maintain a close working 
relationship with group 


4 


12 


11 


22 


4 


4 


1 


0 


15. 


leaders or directors 
To keep up-to-date on 


15 


45 


4 


6 


2 


2 


0 


0 


16. 


materials in his field 
To keep people with whom 
he works informed on what 


18 


54 


2 


4 


0 


0 


0 


0 


happens 

Characteristics 

1. Be dynamic and cause things 


14 


42 


2 


4 


4 


4 


0 


0 


2. 


to happen 

Be a good public relations 


5 


15 


10 


20 


5 


5 


0 


0 


3. 


representative 

Be able to determine the 

interests and creativeness 


6 


18 


14 


28 


0 


0 


0 


0 




of personnel 


9 


27 


6 


12 


5 


5 


0 


0 


4. 

5. 


Be practical 

Be able to maintain rapport 


14 


42 


5 


10 


1 


1 


0 


0 




within the group 


12 


36 


4 


8 


4 


4 


0 


0 


6. 


Be democratic 


9 


27 


7 


14 


4 


4 


0 


0 


7. 

8. 


Be sympathetic 

Have expertise relative to 


4 


12 


10 


20 


6 


6 


0 


0 




his subject matter 


10 


30 


8 


16 


2 


2 


0 


0 


9. 


Be able to accept people 


14 


42 

111- 


6 


12 


0 


0 


0 


0 



3. Frequency of activities of the consultant as viewed by the 

consul tor* 



Activities 


Number Selecting Each Choice 

Practically 

Often Sometimes Never 

No. Score No. Score No. Score 


No 

answer 

No. 


1. 


Make visits to the area where 
the problem developed 


8 


16 


12 


12 


0 


0 


0 


2. 


Help in the pre-planning and 
setting up of programs or plans 


11 


22 


9 


9 


0 


0 


0 


3c 


Conduct or assist in evalua- 
tion of programs or plans 


9 


18 


10 


10 


1 


0 


0 


4. 


Ask for an evaluation of his 
own consultant service* 


3 


6 


12 


12 


5 


0 


0 


5. 


Give suggestions when requested 


14 


28 


6 


6 


0 


0 


0 


6. 


Help to keep projects or plans 
moving toward their objectives 


15 


30 


5 


5 


0 


0 


0 


7. 


Collect and recommend resources 
for use by people being served 


16 


32 


4 


4 


0 


0 


0 


8. 


Listen to problems and plans of 
group or individual participants 


16 


32 


4 


4 


0 


0 


0 


9. 


Criticize plans or procedures* 


2 


4 


17 


17 


1 


0 


0 


39. 


Analyze and interpret the data 
cr information involved 


9 


18 


9 


9 


1 


0 


0 


11. 


Stimulate thinking on problems 
and plans 


11 


22 


8 


8 


0 


0 


0 


12. 


Help organize planning groups 


5 


10 


12 


12 


3 


0 


0 


13. 


Help organize plans of action 


4 


8 


15 


15 


1 


0 


0 


14. 


Demonstrate an idea for clari- 
fication 


4 


8 


14 


14 


1 


0 


1 


15. 


Help to decrease friction within 
groups or among individuals 


4 


8 


14 


14 


2 


0 


0 


16. 


Participate in group 
discussions 


5 


10 


14 


14 


0 


0 


0 


17. 


Help clarify group or indivi- 
dual thinking 


12 


24 


8 


8 


0 


0 


0 


h- 1 

00 

• 


Synthesize materials in order 
to make recommendations (pull 
things together) 


7 


14 


12 


12 


0 


0 


1 


19. 


Interpret the developments 
within a meeting to all members 
involved 


4 


8 


15 


15 


1 


0 


0 


20. 


Prepare for visits before seeing 
the people involved 


14 


28 


5 


5 


0 


0 


1 



* Not Accepted 
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AW EXAMPLE OF THE USE OF 
A VOCATIONAL EDUCATION INFORMATION INVENTORY 

By 

Floyd McKinney* 



Summary 

A sixty- item instrument was designed with the original purpose of measur- 
ing change which might occur as a result of the Evaluation Systems Project 
having been in operation in a local school system Due to delays in the 
development of the instrument this measurement of change was not possible. 
However, the instrument was provided to school systems participating in the 
project, with the suggestion that it might be used to measure the agreement 
or lac it of agreement, concerning the understand ing of vocational education 
between certain groups in their school systems The data presented in this 
report represent the results cf the trial in one school in the use of the 
Vocational Education Information Inventory. 

Returns were received from 104 teachers of vocational education and 18 
administrators in the system A t-test was computed for each item to deter- 
mine if there were significant differences between administrators and teach- 
ers Items were considered significant at: the 05 level of significance. 

Fhe administrators and teachers agreed with the panel of judges on all items, 
although the extent of agreement varied from item to item, 

fifteen of the items were found to show a significant difference in the 
understanding or attitude of teachers as compared to administrators The 
administrators had a lower average score on all of these 15 items than the 
teachers. The items showing a significant difference at the 05 level are 

as. follows: 

4 Vocational education programs tend to result in higher national per 
capita incomes. 



* Research Assistant R & D Unit, Evaluation Systems Project, M.S.U, 
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5. Vocational education increases the employment rate of workers 

*15. Cooperative work experience programs contribute little to the effec 
tivenes3 of vocational education programs „ 

17. Vocational education should emphasize the development of abilities 
that can be used for many years . 

*18 The schools need not accept the responsibility of teaching students 
how to write a letter of application for a job. 

19. Vocational education should prepare its clientele for useful employ 
ment . 

24. Manpower needs can be partially met through vocational education. 

*30. Employer and employee organizations should not expect to be invited 
to cooperate with the school in planning its vocational education 
programs 

31 Mechanization increases the need for specialized training, 

33. Manpower needs of the community should be considered in determining 
the type of vocational education programs to offer 

*36. Simulated experiences are as successful as on-the-job occupational 
experiences in training for employment. 

38- A guidance program should include effective fcllow-up services. 

49 =, Many women need vocational education experiences to vitalize their 
cld skills and to learn new skills . 

*56 If a person is properly trained he can do well in any occupation. 

59. People should be satisfied with, and be making progress in their 
jobs as a result of their vocational education experiences. 

Background 

As the Evaluation Systems Project progressed through its various stages 
it became evident that some attempt, should be made to measure the change 
which might occur among teachers and administrators at the local school 
level as a result of the project having been in operation within the school 
system, One of the evaluative techniques developed for this purpose was a 
"Vocational Education Information Inventory." The Inventory was developed 
after the project was well under way in local schools, thus no measurement 
of the individuals could take place prior to the initiation of the project. 
The data presented in this report are from one school system that used the 
Inventory close to the end of the project year. This report is made in an 
effort to show how the data secured from the Inventory might be used by a 

The panel of judges disagreed with these items. 
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local school system and/or researchers wishing to measure change on the 

part of certain project participants and those directly or indirectly 
affected by a project 



Problem 

The prospect of additional financial assistance, and the status associated 
with a state or nationally recognized research project are sources of motiva- 
tion for many school systems. The appeal to some is also a realization of the 
need for change, or redirection of programs. The question that is extremely 
difficult to answer is the extent of change which results from participation 
in a research project by a local school system. 

One of the more important changes which might be brought about by a proj- 
ect such as this, is a change in the understandings (or attitudes) of those 
individuals teaching and administering the local program of vocational educa- 
tion. This study was an attempt to measure if such a change had taken place. 

Procedures 

Development of Instrument. Items for the Vocational Education Inventory 
were developed by a project research assistant. 1 The items were derived 
from a review of literature and the experiences of the project personnel. 
Following several reviews of the instrument by the project director and 
research assistants the Inventory was submitted to a panel of vocational 
teacher educators and consultants. A panel of 19 judges gave suggestions 
for the improvement and addition or deletion of certa.n items. 

The members of the panel of judges also marked their choices for each 
item according to the following scales 

SA Strongly Agree or Accept 
A Agree or Accept 
U Undecided 
D Disagree 
SD Strongly Disagree 

One of the schools participating in the project used the instrument in con- 
junction with a vocational education evaluation credit workshop which was 

J Floyd McKinney, with advice from Harold Byram, Project Leader 
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part of their local evaluation effort. The results Qf this trial run of the 
instrument were most helpful in identifying questionable items. The revised 
instrument was then submitted to the U # S. Office of Education for approval. 

The instrument itself was arranged so that it could be answered on a 
mechanically scored answer sheet. From these sheets computer cards could be 
automatically produced for data analyzation and for ease of storage. 

Population . When the idea of an instrument to measure change in the under- 
standings and information of project participants was first conceived it was 
hoped that all of the local schools connected with the Project would make use 
of the instrument. Delays in getting the instrument to these schools resulted 
in scattered usage, but the experience of one school and an analyzation of 
the data herein does provide an example which can be utilized in future 
studies. Project personnel had hoped the population would include all voca- 
tional education personnel, administrators, and in some cases perhaps all 
teachers in a local school system. 

Design . These steps were followed in conducting the study: 

a) A review of literature was made to secure appropriate items for 
instrument. 

b) The instrument was critiqued by the project leader and research 
assistants. 

c) A trial-run of the instrument was made at one of the participating 
schools. 

d) A panel of judges was selected to review and rate the items. 

e) Approval of the instrument was secured from the U.S. Office of 
Education. 

f) Local schools were given the option of using the instrument. Most 
of the project schools requested copies. 

g) Answer sheets were obtained from one school so that the data 
could be presented and analyzed. 

Analysis . For those items on the instrument with which the panel of 
judges agreed, the following scores were assigned: SA=5, A-4, U*3, D*2, and 

SD*1. For those items with which the majority of the judges disagreed, 
scale scores were reversed to provide comparable responses for computation 
purposes. An average score was then secured for each item as marked by the 
teachers and administrators. 
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Significance of differences between teachers and administrators was 
determined by use of the t-test. Items were considered significant at the 
.05 level. The formula used in computing the t-test for differences between 
uncorrelated means was: 



t = 




Where and - means of the two samples 
2 2 

and Zx^ = sums of squares in the two samples 
and = numbers of cases in the two samples 
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APPENDIX A 



Comparison of Teacher-Administrator Viewpoint 
Toward Items on ^be Vocational Education Information Inventory 



1 

*1 

4 

*4 

5 

*5 

6 

*6 

7 

*7 

8 

*8 

9 

*9 

10 

*10 

11 

*11 

12 

*12 

13 
*13 

14 
*14 

15 
*15 

16 
*16 

17 
*17 

18 
* 18 



SA 

6 

0 

30 

2 

48 

6 

3 
0 

6 

1 

4 
0 

34 

7 

6 

1 

18 

1 

0 

1 

15 

3 

14 

1 

1 

2 

74 

10 

51 

7 

1 

0 



Number of Responses 



A 


U 


D 


SD 


37 


17 


35 


9 


7 


3 


7 


1 


38 


27 


7 


2 


5 


10 


1 


0 


46 


5 


5 


0 


7 


3 


0 


2 


0 


2 


22 


77 


0 


0 


3 


15 


7 


2 


44 


45 


1 


1 


7 


8 


4 


4 


42 


50 


2 


0 


8 


8 


39 


12 


15 


4 


5 


2 


3 


1 


7 


5 


27 


59 


2 


0 


5 


10 


57 


21 


8 


0 


10 


7 


0 


0 


5 


3 


33 


63 


1 


0 


9 


7 


54 


12 


17 


6 


** 

/ 


4 


3 


1 


21 


30 


24 


15 


5 


8 


2 


2 


2 


8 


33 


60 


1 


1 


7 


7 


23 


6 


1 


0 


7 


0 


0 


1 


42 


6 


3 


3 


4 


3 


4 


0 


0 


0 


26 


77 


2 


0 


5 


11 



per Item 

( 2 ) 

Average Score t-test (3) 

3.03 
3.11 

3.97 
3.44 

4.31 

3.83 

4.65 

4.83 

4.10 

4.11 

4.25 
4.22 

3.80 
3.77 

4.21 
4.16 

3.81 

3.66 

4.48 
4.11 

3.52 
3.44 

3. 04 
2.94 

4.43 
3.33 

4.63 
4.38 

4.32 
3-77 

4.71 
4.38 



2.21 

2 18 



2.02 

2.50 

2.10 
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V o 

ERLC 

I 



Number of Responses 



( 1 ) 



per Item 
( 2 ) 
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SA 


A 


U 


D 


SD 


Average 


19 


78 


26 


0 


0 


0 


4.75 


*19 


10 


6 


1 


0 


1 


4.33 


20 


73 


31 


0 


0 


0 


4.70 


*20 


11 


7 


0 


0 


0 


4.61 


21 


3 


41 


18 


28 


14 


4.21 


*21 


0 


6 


6 


5 


1 


2.94 


22 


8 


31 


25 


33 


7 


3.00 


*22 


1 


2 


8 


4 


3 


3.33 


23 


79 


22 


2 


0 


1 


4.71 


*23 


12 


6 


0 


0 


0 


4.66 


24 


60 


37 


6 


1 


0 


4.50 


*24 


4 


13 


0 


1 


0 


4.11 


25 


28 


33 


21 


20 


2 


3.62 


*25 


5 


5 


4 


3 


1 


3.77 


26 


55 


43 


2 


1 


3 


4.41 


*26 


6 


9 


1 


2 


0 


4.05 


27 


76 


24 


1 


2 


1 


4.65 


*27 


14 


3 


0 


0 


1 


4.61 


28 


4 


3 


5 


46 


46 


4.22 


*28 


1 . 


1 


2 


9 


5 


3.88 


30 


0 


0 


2 


33 


69 


4.64 


*30 


2 


1 


0 


4 


11 


4.16 


31 


66 


36 


0 


1 


1 


4.58 


*31 


5 


11 


0 


2 


0 


4.05 


32 


0 


2 


10 


34 


58 


4.42 


*32 


1 


0 


1 


7 


9 


4.27 


33 


52 


46 


3 


1 


2 


4.39 


*33 


5 


10 


1 


1 


1 


3.94 


34 


4 


6 


5 


39 


50 


4.21 


*34 


0 


2 


0 


9 


7 


4.16 


35 


18 


33 


20 


23 


10 


3.25 


*35 


0 


7 


4 


7 


0 


3.00 


36 


2 


18 


18 


51 


15 


3.75 


*36 


2 


3 


5 


6 


2 


3.16 


37 


55 


38 


8 


2 


1 


4.38 


*37 


9 


7 


0 


0 


2 


4.17 


38 


70 


30 


4 


0 


0 


4.63 


*38 


10 


7 


0 


0 


1 


4.16 



t-test (3) 
3.00 



2.33 



2,66 

3.01 



2.57 



2.26 



2.93 






r. 
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SA 


A 


Number 

U 


of Responses 
D 


(1) 

SD 


per Item 

(2) 

Average 


39 


28 


33 


4 


21 


18 


3.31 


*39 


•V 


6 


4 


4 


1 


3.33 


40 


62 


34 


7 


1 


0 


4- 52 


*40 


11 


6 


0 


1 


0 


4.50 


41 


1 


4 


6 


46 


47 


4.28 


*41 


0 


2 


1 


10 


5 


4 e 00 


43 


41 


51 


7 


3 


2 


4.22 


*43 


2 


14 


1 


1 


0 


3.94 


44 


2 


13 


23 


41 


25 


3.71 


*44 


0 


l 


5 


10 


2 


3.72 


45 


42 


46 


10 


4 


1 


4.16 


*45 


6 


10 


2 


0 


0 


4,22 


46 


36 


50 


15 


3 


0 


4.14 


*46 


5 


11 


1 


1 


0 


4.11 


47 


90 


13 


1 


0 


0 


4.85 


*47 


13 


5 


0 


0 


0 


4.72 


48 


54 


36 


7 


7 


0 


4.31 


*48 


7 


9 


1 


1 


0 


4.22 


49 


52 


43 


5 


4 


0 


4.37 


*49 


4 


10 


3 


i 


0 


3.94 


50 


9 


24 


21 


40 


10 


3.17 


*50 


0 


6 


2 


10 


0 


3.22 


5i 


1 


4 


3 


20 


76 


4.59 


*51 


2 


0 


1 


5 


10 


4.16 


52 


2 


14 


30 


39 


19 


3.56 


*52 


0 


5 


6 


6 


1 


3.16 


53 


13 


31 


18 


30 


12 


3.02 


*53 


1 


6 


3 


7 


1 


2,94 


54 


57 


34 


5 


7 


1 


4.33 


*54 


7 


10 


1 


0 


0 


4, 33 


55 


19 


46 


26 


10 


3 


3.65 


*55 


1 


8 


6 


2 


1 


3.33 


56 


5 


20 


14 


39 


26 


3.58 


*56 


0 


3 


4 


9 


2 


3.00 


57 


4 


27 


21 


37 


15 


3.31 


*57 


1 


4 


6 


7 


0 


3.05 


58 


5 


14 


12 


56 


17 


3,63 


*58 


0 


1 


6 


10 


1 


3,61 



422- 



41 



t-test (3) 



2.26 



2 00 




Number of Responses 



( 1 ) 



per Item 
( 2 ) 





SA 


A_ 


U 


D 


SD 


Average 


59 


17 


58 


14 


10 


5 


4.65 


*59 


1 


12 


4 


1 


0 


3.72 


60 


20 


42 


17 


23 


2 


2.47 


*60 


2 


9 


5 


2 


0 


2.38 



t-test (3) 



3.67 



* Numbers with asterisk represent administrators, 
Numbers without asterisk represent teachers. 



(1) Scale: 



SA 

A 

U 

D 

SD 



Strongly Agree or Accept 
Agree or Accept 
Undecided 
Disagree 

Strongly Disagree 



(2) Average Score Bases: 

SA = 5, A = 4, U = 3, D = 2, SD * 1 

For those items of the inventory with which the majority of the 
judges disagreed, scale scores were reversed to provide comparable 
responses for computation purposes. 



(3) Computed for the .05 level of significance, 
listed did not reach the .05 level. 



Items with no t-test score 



(4) Items 2, 3, 29, and 42 were deleted from the analysis because of lack of 
agreement among the panel of judges as to the most appropriate response 
to the item. 
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RESEARCH AND DEVELOPMENT PROGRAM 
IN VOCATIONAL- TECHNICAL EDUCATION 



Michigan State University 
College of Education 

EVALUATION SYSTEMS PROJECT 

VOCATIONAL EDUCATION INFORMATION INVENTORY 

Directions: Be sure your name and sex is indicated on the response sheet. 

Carefully read each statement and mark your reaction on the 
response sheet. React to each statement by marking the cate- 
gory which most nearly corresponds to your own beliefs. 

1 - I strongly agree with, or accept the statement 
2-1 tend to agree with, or accept the statement 
3 - I am undecided in regard to the statement 

4- 1 tend to disagree with, or reject the statement 

5- 1 strongly disagree with, or reject the statement 

1. General education is the best preparation for work. 

2. Vocational education should be confined to occupations in which the 
students cau secure a job when they leave or graduate from school. 

3. The best job training is that given prior to employment. 

4. Vocational education programs tend to result in higher national oer 

capita incomes. p 

5. Vocational education increases the employment rate of workers. 

6. A course that helps students improve their personal and social living 
habits has no place in the vocational education curriculum. 

7. Exploratory values to a student should not be expected In courses that 
are a part of a vocational education sequence. 

8. Students at the high school level are too immature to make it worth- 
while to provide the specialized courses to prepare them for specific 
occupations. 

9. All people can be trained for a level of work commensurate with their 
level of abilities. 
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10. Vocational education is largely for those unable to be academically 
successful . 



11* Many of the traditional areas in vocational education could combine cer- 
tain of their basic courses so that basic principles and concepts could 
be taught in a class composed of students with occupational goals commonly 
thought of as representative of different vocational areas ^ 

12. Vocational education should concern itself with only the development of 
skills. 



13. An important function of vocational education is teaching people to work 
with their hands. 



14. Occupational-cluster training (examples: Health Services, Electronics) 
should be given on the junior high school level. 



15. Cooperative work experie 
ness of vocational educ* 



-ograms contribute little to the effective 
rograms . 



16. Vocational education _ 
techniques of being inter 



..lude the preparation of students in the 

.d. 



17. Vocational education should emphasize the development of abilities that 
can be used for many years . 



18. The school need not accept the responsibility of teaching students how to 
write a letter of application for a job. 

19. Vocational education should prepare its clientele for useful employment. 



20. Students should learn how to apply in person for a job as a result of 
th^ir experiences in vocational education. 



21. The vocational education student is generally one of lower s^cio-econ- 
cmic status. 

22. Employers will provide training needed by most of their workers „ 



23. Learning is one of the goals of vocational education. 

24„ Manpower needs can be partially met through vocational education. 

25. Youth should possess a marketable occupational skill when they leave 
high school. 

26. Cooperative education programs provide good opportunities for youth to 
develop their vocational skills. 

27. Educators need to be cognizant cf the entrance requirements in the occu 
pations for which they are educating persons. 




“ 125 - 




28 



29. 

30. 

31 

32 

33 
34. 



35 



36 c 



37, 



38. 

39. 

40. 

41. 

42. 



The types of vocational courses offered in a school need not bear 
relationsn\p to the inflow and outflow of workers. 



a 



Vocational education should generally be related 
tunities in local areas. 



to employment oppor- 



Employer and employee organizations should not expect to be invited to 
cooperate -with the school in planning its vocational education programs 

Mechanization increases the need for specialized training. 



Persons in charge of vocational education need not feel 
bixity in the education of the handicapped. 



any responsi 



Manpower needs of the community should be considered 
type of vocational education programs to offer 



in determining the 



Specific skill training should not be offered in high school. 



Junior high school students lack the maturity and experience 
from specialized vocational courses. 



to profit 



Simulated experiences are as successful as on-the-job occupational 
experiences in training for employment. 



Schools have a responsibility to provide occupational education for 
persons of all levels of ability 



A guidance program should include effective follow-up services 



Vocational 
to prepare 



courses are as valuable as any other courses in the curriculum 
students to succeed as college or university students. 



Information about 
maintained by the 



student work experience should be a part of the records 
guidance office. 



Effective guidance can be provided without access to the information 
found m an occupational survey of the labor needs of the area 



Guidance is generally more necessary at the adolescent 
adult level. 



level than at the 



43. School guidance programs should assist students in making a tentative 
occupational choice, 



44 



It is not a responsibility 
Student placement services 
agencies. 



the school to provide placement services 4 
are adequately provided by governmental 



45. 



It is important to consider the nations balance between labor 
and surplus when counseling students about occupations. 



shortage 
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46 . 



47. 

48. 

49. 

50. 

51. 
52.. 

53. 

54. 

55. 

56c 

57. 

58. 

59. 

60 c 



Optimum utilization of human resources is one measure of an effective 
guidance program. 

Occupational information should be available to all students. 

Vocational education for adults should include specific training for 
specific jobs. 



Many women need vocational education experiences to vitalize their old 
skills and to learn new skills. 



The upgrading of the skills of adults is more the responsibility of 
employers than of the educational system. 

All vocational education should be conducted as a post-high ..ool 
program. 

Vocational education for all adults yields dividends to the community 
more slowly than similar instruction for youth in the secondary school. 

The best time to teach job skills is when they are needed. 

Educating persons for a vocation results in as great a return to 
society as educating persons for the professions. 

As the amount of schooling an individual receives decreases, the need 
for specialization in his vocational preparation increases. 

If a person is properly trained he can do well in any occupation. 

The proportion of people who engage in an occupation in a field in which 
they were trained, is a poor measure of the training effectiveness of a 
program. 

Educators should be more concerned with educating future workers than 
with on-the-job success of their former students. 



People should be satisfied with, and be making progress in their jobs 
as a result of their vocational education experiences. 

Students who enroll in vocational education courses at the secondary 
level are not as well prepared for college as those enrolled in a college 
preparatory curriculum* 
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